Oct 31 07:26:32.748792 ip-10-0-138-234 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 31 07:26:32.748803 ip-10-0-138-234 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 31 07:26:32.748810 ip-10-0-138-234 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 31 07:26:32.749063 ip-10-0-138-234 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 31 07:26:42.423215 ip-10-0-138-234 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Oct 31 07:26:42.423235 ip-10-0-138-234 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 5bbc108c7a5646dc9d1143297cde19dc --
Oct 31 07:28:36.414793 ip-10-0-138-234 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 07:28:36.782103 ip-10-0-138-234 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 07:28:36.782103 ip-10-0-138-234 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 07:28:36.782103 ip-10-0-138-234 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 07:28:36.782103 ip-10-0-138-234 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 07:28:36.785722 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785635    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:36.785722 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785721    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:36.785722 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785725    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785729    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785733    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785735    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785738    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785742    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785745    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785747    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785750    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785753    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785755    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785758    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785761    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785764    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785766    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785769    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785771    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785774    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785776    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:36.785842 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785779    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785782    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785784    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785789    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785795    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785799    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785802    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785804    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785807    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785810    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785813    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785817    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785821    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785825    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785828    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785831    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785835    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785837    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785840    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:36.786294 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785843    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785846    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785849    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785851    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785854    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785857    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785859    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785862    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785864    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785867    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785870    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785873    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785875    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785878    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785880    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785883    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785885    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785888    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785891    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:36.786765 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785893    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785896    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785899    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785901    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785904    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785907    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785910    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785913    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785916    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785919    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785922    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785924    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785927    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785929    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785932    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785934    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785937    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785939    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785942    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785951    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785954    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:36.787220 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785957    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785959    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785962    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785972    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785975    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785978    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785980    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785983    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785986    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785989    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785991    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785994    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785996    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.785999    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786002    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786004    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786007    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786010    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786012    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:36.787736 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786015    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786018    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786021    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786024    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786027    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786030    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786032    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786833    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786840    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786843    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786846    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786849    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786852    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786855    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786857    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786862    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786866    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786873    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786876    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:36.788184 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786880    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786882    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786885    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786887    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786890    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786893    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786896    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786898    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786901    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786904    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786906    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786909    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786912    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786915    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786918    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786920    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786923    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786926    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786928    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:36.788682 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786933    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786937    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786940    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786942    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786945    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786948    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786950    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786953    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786955    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786958    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786960    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786963    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786966    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786969    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786971    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786973    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786976    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786979    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786981    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786984    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:36.789143 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786987    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786989    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786992    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786995    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.786998    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787001    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787004    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787006    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787009    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787012    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787014    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787017    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787020    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787022    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787025    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787027    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787030    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787032    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787035    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:36.789627 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787037    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787040    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787042    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787044    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787047    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787050    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787053    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787056    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787058    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787061    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787063    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787066    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787070    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787073    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787075    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787078    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787081    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787083    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787086    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787088    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:36.790076 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787091    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787093    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787096    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787098    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787101    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787104    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787106    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787108    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787111    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787113    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787116    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787118    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787121    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787123    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787125    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787128    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787628    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787634    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787637    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787646    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:36.790563 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787650    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787653    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787656    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787659    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787662    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787665    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787667    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787670    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787673    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787675    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787678    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787681    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787686    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787689    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787692    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787695    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787697    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787700    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787702    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:36.791036 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787705    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787708    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787710    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787712    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787715    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787718    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787720    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787723    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787725    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787727    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787730    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787732    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787735    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787738    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787741    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787743    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787746    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787749    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787752    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787755    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:36.791532 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787757    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787759    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787762    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787764    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787767    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787770    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787772    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787775    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787777    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787781    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787783    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787786    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787788    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787791    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787793    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787796    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787799    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787801    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787803    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787809    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:36.792023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787812    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787814    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787818    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787821    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787824    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787828    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787831    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787833    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787836    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787838    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787841    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787844    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787846    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787849    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787851    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787854    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787856    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787859    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787862    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:36.792529 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787865    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787867    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787870    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787873    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787875    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787878    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787880    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787883    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787885    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787888    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787890    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787893    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787897    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787899    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787902    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787905    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787907    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787910    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787912    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:36.793017 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787915    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787918    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787920    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787924    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.787926    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788509    2572 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788519    2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788526    2572 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788530    2572 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788535    2572 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788539    2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788545    2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788549    2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788553    2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788556    2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788559    2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788562    2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788566    2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788569    2572 flags.go:64] FLAG: --cgroup-root=""
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788572    2572 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788575    2572 flags.go:64] FLAG: --client-ca-file=""
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788577    2572 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788580    2572 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788585    2572 flags.go:64] FLAG: --cluster-domain=""
Oct 31 07:28:36.793471 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788588    2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788593    2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788596    2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788599    2572 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788603    2572 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788606    2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788610    2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788613    2572 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788616    2572 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788619    2572 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788622    2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788626    2572 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788629    2572 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788633    2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788636    2572 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788639    2572 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788642    2572 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788645    2572 flags.go:64] FLAG: --enable-server="true"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788648    2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788654    2572 flags.go:64] FLAG: --event-burst="100"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788657    2572 flags.go:64] FLAG: --event-qps="50"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788660    2572 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788663    2572 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788666    2572 flags.go:64] FLAG: --eviction-hard=""
Oct 31 07:28:36.794048 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788670    2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788673    2572 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788676    2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788679    2572 flags.go:64] FLAG: --eviction-soft=""
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788682    2572 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788685    2572 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788688    2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788691    2572 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788694    2572 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788698    2572 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788700    2572 flags.go:64] FLAG: --feature-gates=""
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788705    2572 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788708    2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788711    2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788714    2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788717    2572 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788720    2572 flags.go:64] FLAG: --help="false"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788723    2572 flags.go:64] FLAG: --hostname-override="ip-10-0-138-234.ec2.internal"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788733    2572 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788737    2572 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788740    2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788743    2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788746    2572 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788749    2572 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 07:28:36.794728 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788752    2572 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788755    2572 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788758    2572 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788762    2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788765    2572 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788768    2572 flags.go:64] FLAG: --kube-reserved=""
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788771    2572 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788774    2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788777    2572 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788780    2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788783    2572 flags.go:64] FLAG: --lock-file=""
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788786    2572 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788789    2572 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788792    2572 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788798    2572 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788801    2572 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788804    2572 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788807    2572 flags.go:64] FLAG: --logging-format="text"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788810    2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788814    2572 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788817    2572 flags.go:64] FLAG: --manifest-url=""
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788819    2572 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788824    2572 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788827    2572 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788831    2572 flags.go:64] FLAG: --max-pods="110"
Oct 31 07:28:36.795305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788834    2572 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788837    2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788841    2572 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788844    2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788847    2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788850    2572 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788853    2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788860    2572 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788863    2572 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788866    2572 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788870    2572 flags.go:64] FLAG: --pod-cidr=""
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788873    2572 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788876    2572 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788879    2572 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788882    2572 flags.go:64] FLAG: --port="10250"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788885    2572 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788888    2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f35e7a779a2e043e"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788891    2572 flags.go:64] FLAG: --qos-reserved=""
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788894    2572 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788897    2572 flags.go:64] FLAG: --register-node="true"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788900    2572 flags.go:64] FLAG: --register-with-taints=""
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788904    2572 flags.go:64] FLAG: --registry-burst="10"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788907    2572 flags.go:64] FLAG: --registry-qps="5"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788910    2572 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788913    2572 flags.go:64] FLAG: --reserved-memory=""
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788917    2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 07:28:36.795917 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788920    2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788923    2572 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788926    2572 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788928    2572 flags.go:64] FLAG: --runonce="false"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788931    2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788934    2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788937    2572 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788940    2572 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788943    2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788949    2572 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788953    2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788956    2572 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788959    2572 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788962    2572 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788965    2572 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788968    2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788972    2572 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788975    2572 flags.go:64] FLAG: --system-cgroups=""
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788978    2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788983    2572 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788986    2572 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788989    2572 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788993    2572 flags.go:64] FLAG: --tls-min-version=""
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788996    2572 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.788998    2572 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 07:28:36.796616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.789001    2572 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.789004    2572 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.789007    2572 flags.go:64] FLAG: --v="2"
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.789011    2572 flags.go:64] FLAG: --version="false"
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.789015    2572 flags.go:64] FLAG: --vmodule=""
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.789019    2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.789022    2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789157    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789161    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789165    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789169    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789173    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789176    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789179    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789183    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789186    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789189    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789193    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789196    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789199    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789201    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:36.797203 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789204    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789207    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789210    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789213    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789215    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789217    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789220    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789222    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789225    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789227    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789230    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789232    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789235    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789238    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789240    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789244    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789247    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789250    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789252    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:36.797721 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789255    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789257    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789259    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789263    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789265    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789268    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789271    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789273    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789275    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789278    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789281    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789284    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789286    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789289    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789291    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789294    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789296    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789298    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789302    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789304    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:36.798162 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789307    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789309    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789312    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789314    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789317    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789319    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789321    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789324    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789328    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789330    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789333    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789335    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789338    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789340    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789343    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789345    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789348    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789350    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789353    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:36.798665 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789356    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789358    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789361    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789363    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789367    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789369    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789372    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789374    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789377    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789379    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789382    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789384    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789387    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789389    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789392    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789394    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789397    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789399    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789402    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789404    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:36.799135 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789406    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789410    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789413    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789415    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789418    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789420    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789423    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789425    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789428    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789430    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789433    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789435    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789438    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789440    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.789479    2572 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789514    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:36.799618 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789518    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789522    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789525    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789528    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789531    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789533    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789536    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789538    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789541    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789543    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789546    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789548    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789551    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789553    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789556    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789558    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789561    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789564    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789567    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789570    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:36.799999 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789572    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789574    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789577    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789579    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789582    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789584    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789587    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789589    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789592    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789594    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789596    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789599    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789602    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789605    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789609    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789611    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789614    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789617    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789620    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789624    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:36.800545 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789628    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789631    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789634    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789637    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789641    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789645    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789648    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789651    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789654    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789658    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789660    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789663    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789665    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789668    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789670    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789673    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789675    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789678    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789680    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:36.801023 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789683    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789685    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789688    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789690    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789693    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789695    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789699    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789702    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789705    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789707    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789710    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789712    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789715    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789717    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789720    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789722    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789725    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789727    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789730    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789732    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:36.801478 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789735    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789737    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789741    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789744    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789746    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789749    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789751    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789754    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789756    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789759    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789762    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789764    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789767    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789769    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789772    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789774    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789777    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789779    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789782    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789785    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:36.801993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789787    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789790    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789798    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789801    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789803    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.789805    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.796074    2572 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.796089    2572 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.796104    2572 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.796109    2572 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796165    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796170    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796174    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796180    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796185    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796188    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796191    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796194    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796197    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:36.802477 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796200    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796202    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796205    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796208    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796210    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796213    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796215    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796218    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796221    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796223    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796226    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796229    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796231    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796234    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796236    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796239    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796242    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796244    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796247    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:36.802925 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796249    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796253    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796255    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796258    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796261    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796265    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796269    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796272    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796275    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796277    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796280    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796282    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796285    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796287    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796290    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796293    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796295    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796297    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796300    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:36.803373 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796303    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796306    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796308    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796311    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796314    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796316    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796319    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796321    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796324    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796327    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796329    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796332    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796334    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796337    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796339    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796342    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796345    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796347    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796350    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796352    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:36.803951 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796355    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796358    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796360    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796363    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796365    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796368    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796370    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796373    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796375    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796377    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796380    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796383    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796385    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796388    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796390    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796393    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796395    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796398    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796400    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:36.804800 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796402    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796405    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796408    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796410    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796413    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796415    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796417    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796420    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796422    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796425    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796428    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796430    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796433    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796435    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796438    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796441    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796443    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796445    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796467    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:36.805422 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796470    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.796500    2572 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796641    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796645    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796649    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796652    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796654    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796657    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796661    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796666    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796669    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796672    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796676    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796679    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796682    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796684    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:36.805977 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796687    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796690    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796693    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796696    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796699    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796703    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796705    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796708    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796711    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796714    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796717    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796719    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796722    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796724    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796727    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796729    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796732    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796734    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796737    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796739    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:36.806395 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796742    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796744    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796747    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796749    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796752    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796754    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796757    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796759    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796762    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796764    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796767    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796770    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796772    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796775    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796777    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796780    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796782    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796785    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796787    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796790    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:36.806887 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796793    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796795    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796798    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796801    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796803    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796806    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796808    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796810    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796813    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796815    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796818    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796820    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796823    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796825    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796828    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796830    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796833    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796835    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796841    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796843    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:36.807349 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796846    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796848    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796851    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796854    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796856    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796859    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796861    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796863    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796867    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796870    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796872    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796875    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796877    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796880    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796883    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796885    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796888    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796891    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796893    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:36.807869 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796896    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796898    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796901    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796904    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796906    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796909    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796911    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796913    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796916    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796918    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796921    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796926    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796928    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.796951    2572 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796983    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796987    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:36.808605 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796990    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796993    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796995    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.796998    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797000    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797003    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797006    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797009    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797012    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797014    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797017    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797020    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797023    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797026    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797028    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797031    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797034    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797036    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797039    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:36.808993 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797041    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797044    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797047    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797049    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797052    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797054    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797057    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797059    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797063    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797066    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797069    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797071    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797074    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797076    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797079    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797081    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797084    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797086    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797090    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:36.809433 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797095    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797098    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797102    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797105    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797108    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797111    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797114    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797116    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797119    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797121    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797124    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797127    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797130    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797132    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797135    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797138    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797140    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797143    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797146    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797148    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:36.810006 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797151    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797154    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797157    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797159    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797162    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797164    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797167    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797169    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797172    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797175    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797177    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797180    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797184    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797186    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797189    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797191    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797194    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797197    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797199    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:36.810486 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797203    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797206    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797209    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797212    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797215    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797217    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797220    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797222    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797225    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797227    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797230    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797233    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797235    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797238    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797241    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797244    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797246    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797249    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797251    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797254    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:36.810944 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797256    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:36.811409 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797259    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:36.811409 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797261    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:36.811409 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797264    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:36.811409 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797267    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:36.811409 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797271    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:36.811409 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:36.797274    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:36.811409 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.797324    2572 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 07:28:36.811409 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.801146    2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 31 07:28:36.811409 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.801994    2572 server.go:1014] "Starting client certificate rotation"
Oct 31 07:28:36.811409 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.802092    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 07:28:36.811409 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.802121    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 07:28:36.811409 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.806094    2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 07:28:36.811409 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.808741    2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 07:28:36.817333 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.817315    2572 log.go:25] "Validated CRI v1 runtime API"
Oct 31 07:28:36.820183 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.820169    2572 log.go:25] "Validated CRI v1 image API"
Oct 31 07:28:36.820976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.820951    2572 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 07:28:36.824734 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.824711    2572 fs.go:135] Filesystem UUIDs: map[1dcdf532-963a-48eb-9e1b-cae16f327670:/dev/nvme0n1p4 55737d7b-3d13-4d35-a487-76872c3424c6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Oct 31 07:28:36.824808 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.824732    2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 31 07:28:36.830731 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.830614    2572 manager.go:217] Machine: {Timestamp:2026-10-31 07:28:36.82841983 +0000 UTC m=+0.310530716 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106554 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c87ff65477149382115d754641c0d SystemUUID:ec2c87ff-6547-7149-3821-15d754641c0d BootID:5bbc108c-7a56-46dc-9d11-43297cde19dc Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f7:04:22:61 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f7:04:22:61 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:52:2b:e0:00:a6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 07:28:36.830731 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.830723    2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 07:28:36.830897 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.830832    2572 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 07:28:36.832816 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.832785    2572 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 07:28:36.832956 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.832820    2572 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-234.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 07:28:36.832998 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.832959    2572 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 07:28:36.832998 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.832967    2572 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 07:28:36.832998 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.832994    2572 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 07:28:36.833080 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.833008    2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 07:28:36.833870 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.833860    2572 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 07:28:36.837438 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.837426    2572 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 07:28:36.837572 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.837562    2572 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 07:28:36.838151 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.838142    2572 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 07:28:36.838180 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.838155    2572 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 07:28:36.838180 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.838165    2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 07:28:36.838180 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.838174    2572 kubelet.go:405] "Adding apiserver pod source"
Oct 31 07:28:36.838290 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.838181    2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 07:28:36.838557 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.838546    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 07:28:36.841315 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.841293    2572 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 07:28:36.845608 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.845594    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 07:28:36.846233 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.846221    2572 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 07:28:36.846280 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.846251    2572 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 07:28:36.846421 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.846411    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 07:28:36.846476 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.846427    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 07:28:36.846476 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.846433    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 07:28:36.846476 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.846439    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 07:28:36.846476 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.846445    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 07:28:36.846476 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.846469    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 07:28:36.847537 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.847527    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 07:28:36.847537 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.847537    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 07:28:36.847593 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.847545    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 07:28:36.847593 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.847550    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 07:28:36.847593 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.847559    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 07:28:36.847593 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.847568    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 07:28:36.848038 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.848028    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 07:28:36.848038 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.848038    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 07:28:36.851581 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.851569    2572 server.go:1263] "Started kubelet"
Oct 31 07:28:36.851702 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.851674    2572 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 07:28:36.851762 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.851673    2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 07:28:36.851762 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.851737    2572 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 07:28:36.852248 ip-10-0-138-234 systemd[1]: Started Kubernetes Kubelet.
Oct 31 07:28:36.852796 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.852777    2572 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 07:28:36.853986 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.853904    2572 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 07:28:36.857814 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.857684    2572 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 07:28:36.857814 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.857711    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 07:28:36.858005 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:36.857979    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:36.858261 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.858229    2572 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 07:28:36.858261 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.858253    2572 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 07:28:36.858400 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.858308    2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 07:28:36.858400 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.858372    2572 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 07:28:36.858400 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.858380    2572 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 07:28:36.859016 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.858995    2572 factory.go:55] Registering systemd factory
Oct 31 07:28:36.859016 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.859017    2572 factory.go:223] Registration of the systemd container factory successfully
Oct 31 07:28:36.859305 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.859291    2572 factory.go:153] Registering CRI-O factory
Oct 31 07:28:36.859343 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.859311    2572 factory.go:223] Registration of the crio container factory successfully
Oct 31 07:28:36.859377 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.859365    2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 07:28:36.859409 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.859383    2572 factory.go:103] Registering Raw factory
Oct 31 07:28:36.859409 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.859398    2572 manager.go:1199] Started watching for new ooms in manager
Oct 31 07:28:36.860277 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.860259    2572 manager.go:319] Starting recovery of all containers
Oct 31 07:28:36.860398 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:36.860342    2572 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 31 07:28:36.873912 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.873893    2572 manager.go:324] Recovery completed
Oct 31 07:28:36.878107 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.878094    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:36.879933 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.879909    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:36.879994 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.879947    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:36.879994 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.879958    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID"
Oct 31 07:28:36.880906 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.880893    2572 cpu_manager.go:226] "Starting" policy="none"
Oct 31 07:28:36.880906 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.880895    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:36.880906 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.880908    2572 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 07:28:36.881044 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.880928    2572 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 07:28:36.881546 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.881532    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:36.881601 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.881558    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:36.881601 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.881568    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID"
Oct 31 07:28:36.883180 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.883168    2572 policy_none.go:50] "Start"
Oct 31 07:28:36.883222 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.883187    2572 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 07:28:36.883222 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.883199    2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 07:28:36.885338 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.885323    2572 policy_none.go:44] "Start"
Oct 31 07:28:36.887253 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.887222    2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-234.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 31 07:28:36.888096 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:36.887234    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-234.ec2.internal.18e38ca4fbdad927  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-234.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-234.ec2.internal,},FirstTimestamp:2026-10-31 07:28:36.851538215 +0000 UTC m=+0.333649100,LastTimestamp:2026-10-31 07:28:36.851538215 +0000 UTC m=+0.333649100,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-234.ec2.internal,}"
Oct 31 07:28:36.898629 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:36.898605    2572 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-234.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 31 07:28:36.899117 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:36.899082    2572 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-234.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 31 07:28:36.910840 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.910822    2572 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 07:28:36.910916 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.910870    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bdsz7"
Oct 31 07:28:36.912067 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.912040    2572 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 07:28:36.916171 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.916157    2572 manager.go:341] "Starting Device Plugin manager"
Oct 31 07:28:36.916260 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:36.916195    2572 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 07:28:36.916260 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.916207    2572 server.go:87] "Starting device plugin registration server"
Oct 31 07:28:36.916416 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.916404    2572 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 07:28:36.916482 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.916421    2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 07:28:36.916575 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.916553    2572 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 07:28:36.916669 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.916632    2572 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 07:28:36.916669 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:36.916639    2572 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 07:28:36.917136 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:36.917115    2572 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 07:28:36.917209 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:36.917169    2572 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:36.940394 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:36.940308    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-234.ec2.internal.18e38ca4fd8c1f78  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-234.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-234.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-234.ec2.internal,},FirstTimestamp:2026-10-31 07:28:36.879933304 +0000 UTC m=+0.362044191,LastTimestamp:2026-10-31 07:28:36.879933304 +0000 UTC m=+0.362044191,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-234.ec2.internal,}"
Oct 31 07:28:36.940518 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:36.940388    2572 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 31 07:28:36.941669 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:36.941649    2572 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 31 07:28:36.992007 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:36.991931    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-234.ec2.internal.18e38ca4fd8c65af  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-234.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-234.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-234.ec2.internal,},FirstTimestamp:2026-10-31 07:28:36.879951279 +0000 UTC m=+0.362062164,LastTimestamp:2026-10-31 07:28:36.879951279 +0000 UTC m=+0.362062164,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-234.ec2.internal,}"
Oct 31 07:28:37.016856 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.016699    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:37.017773 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.017758    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:37.017844 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.017785    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:37.017844 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.017795    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID"
Oct 31 07:28:37.017844 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.017820    2572 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.056736 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.056718    2572 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.073926 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:37.056751    2572 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-234.ec2.internal\": node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:37.098739 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:37.098712    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:37.100744 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.100720    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 07:28:37.101919 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.101903    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 07:28:37.101969 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.101928    2572 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 07:28:37.101969 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.101950    2572 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 07:28:37.102043 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:37.101988    2572 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 31 07:28:37.120134 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.120115    2572 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:28:37.199014 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:37.198986    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:37.202109 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.202091    2572 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"]
Oct 31 07:28:37.202163 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.202148    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:37.203090 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.203077    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:37.203158 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.203102    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:37.203158 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.203112    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID"
Oct 31 07:28:37.204966 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.204955    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:37.205098 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.205084    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.205156 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.205116    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:37.205601 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.205584    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:37.205687 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.205613    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:37.205687 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.205625    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID"
Oct 31 07:28:37.205767 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.205695    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:37.205767 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.205719    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:37.205767 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.205729    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID"
Oct 31 07:28:37.207505 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.207487    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.207583 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.207524    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:37.208222 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.208206    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:37.208296 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.208230    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:37.208296 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.208240    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID"
Oct 31 07:28:37.236875 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:37.236854    2572 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-234.ec2.internal\" not found" node="ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.243192 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:37.243172    2572 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-234.ec2.internal\" not found" node="ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.260774 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.260755    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/700ea6542926bcf6606c986e3906f152-config\") pod \"kube-apiserver-proxy-ip-10-0-138-234.ec2.internal\" (UID: \"700ea6542926bcf6606c986e3906f152\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.260865 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.260788    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee833da619c66751ed2de8e982aff1e5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"ee833da619c66751ed2de8e982aff1e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.260865 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.260816    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee833da619c66751ed2de8e982aff1e5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"ee833da619c66751ed2de8e982aff1e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.299502 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:37.299482    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:37.361382 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.361312    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/700ea6542926bcf6606c986e3906f152-config\") pod \"kube-apiserver-proxy-ip-10-0-138-234.ec2.internal\" (UID: \"700ea6542926bcf6606c986e3906f152\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.361382 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.361352    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee833da619c66751ed2de8e982aff1e5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"ee833da619c66751ed2de8e982aff1e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.361382 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.361374    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee833da619c66751ed2de8e982aff1e5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"ee833da619c66751ed2de8e982aff1e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.361553 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.361409    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/700ea6542926bcf6606c986e3906f152-config\") pod \"kube-apiserver-proxy-ip-10-0-138-234.ec2.internal\" (UID: \"700ea6542926bcf6606c986e3906f152\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.361553 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.361411    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee833da619c66751ed2de8e982aff1e5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"ee833da619c66751ed2de8e982aff1e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.361553 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.361412    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee833da619c66751ed2de8e982aff1e5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"ee833da619c66751ed2de8e982aff1e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.399594 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:37.399561    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:37.500513 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:37.500482    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:37.540715 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.540690    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.546562 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.546548    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:28:37.600612 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:37.600581    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:37.701118 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:37.701040    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:37.801577 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:37.801547    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:37.802111 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.801585    2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 31 07:28:37.802111 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:37.801751    2572 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 31 07:28:39.174817 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:39.174790    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:39.275703 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:39.275674    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:39.288853 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.288805    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 07:23:38 +0000 UTC" deadline="2028-04-24 01:47:07.625904348 +0000 UTC"
Oct 31 07:28:39.288853 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.288851    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12978h18m28.337056863s"
Oct 31 07:28:39.329079 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.329048    2572 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:28:39.346567 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:39.346526    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod700ea6542926bcf6606c986e3906f152.slice/crio-6fff91a9f5dad61d1b0a19109961e99874b1dd5f8faa562d54b9cb23426ae956 WatchSource:0}: Error finding container 6fff91a9f5dad61d1b0a19109961e99874b1dd5f8faa562d54b9cb23426ae956: Status 404 returned error can't find the container with id 6fff91a9f5dad61d1b0a19109961e99874b1dd5f8faa562d54b9cb23426ae956
Oct 31 07:28:39.346804 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:39.346782    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee833da619c66751ed2de8e982aff1e5.slice/crio-bf9240bcbb27028905a2df9dc22ad44f2f83a0db479f52303bd11a643739d80a WatchSource:0}: Error finding container bf9240bcbb27028905a2df9dc22ad44f2f83a0db479f52303bd11a643739d80a: Status 404 returned error can't find the container with id bf9240bcbb27028905a2df9dc22ad44f2f83a0db479f52303bd11a643739d80a
Oct 31 07:28:39.351243 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.351216    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:39.352159 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.352144    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 07:28:39.352262 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.352220    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:39.352303 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.352278    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:39.352303 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.352296    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID"
Oct 31 07:28:39.375981 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:39.375958    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:39.379059 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.379017    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal" event={"ID":"700ea6542926bcf6606c986e3906f152","Type":"ContainerStarted","Data":"6fff91a9f5dad61d1b0a19109961e99874b1dd5f8faa562d54b9cb23426ae956"}
Oct 31 07:28:39.380486 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.380466    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" event={"ID":"ee833da619c66751ed2de8e982aff1e5","Type":"ContainerStarted","Data":"bf9240bcbb27028905a2df9dc22ad44f2f83a0db479f52303bd11a643739d80a"}
Oct 31 07:28:39.468953 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.468929    2572 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:28:39.476976 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:39.476955    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:39.577470 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:39.577412    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:28:39.607907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.607886    2572 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:28:39.631086 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.631066    2572 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal"
Oct 31 07:28:39.654505 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.654408    2572 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 07:28:39.655590 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.655560    2572 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:28:39.684423 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:39.684394    2572 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 07:28:40.112713 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.112684    2572 apiserver.go:52] "Watching apiserver"
Oct 31 07:28:40.130899 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.130869    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 31 07:28:40.160940 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.159832    2572 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 07:28:40.160940 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.160555    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-gkl8q","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal","openshift-multus/multus-jghjq","openshift-network-operator/iptables-alerter-z22fp","kube-system/global-pull-secret-syncer-z9h8d","kube-system/konnectivity-agent-6ll6x","openshift-network-diagnostics/network-check-target-pjddt","openshift-image-registry/node-ca-fl4w7","openshift-multus/multus-additional-cni-plugins-h4qfb","openshift-multus/network-metrics-daemon-2skf7","openshift-ovn-kubernetes/ovnkube-node-vzgl4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l","openshift-cluster-node-tuning-operator/tuned-m7rmk"]
Oct 31 07:28:40.163210 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.163186    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:28:40.165452 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.165425    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.167772 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.167749    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.169891 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.169871    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.173103 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.173086    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:28:40.175396 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.175376    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:40.175490 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.175437    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h8d" podUID="11dc8b0e-286d-42e1-bfe0-d882e7dfe073"
Oct 31 07:28:40.178276 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.177933    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:28:40.180940 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.180366    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 07:28:40.180940 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.180409    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 07:28:40.180940 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.180460    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-ddkhs\""
Oct 31 07:28:40.181705 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.181686    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:28:40.182043 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.182021    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 07:28:40.182115 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.182095    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 07:28:40.182344 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.182326    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 07:28:40.182413 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.182341    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xcjxx\""
Oct 31 07:28:40.184772 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.184753    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:40.184863 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.184821    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjddt" podUID="72245812-04f1-47b8-a22f-67a61f175b63"
Oct 31 07:28:40.187247 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.187212    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mwmjb\""
Oct 31 07:28:40.187508 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.187495    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.189913 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.189896    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:40.190003 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.189949    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2skf7" podUID="d95920d0-5a98-46bc-9b64-f820f02d5b1d"
Oct 31 07:28:40.192647 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.192629    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.193272 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.193238    2572 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 07:28:40.193352 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.193314    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ghk54"
Oct 31 07:28:40.232125 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.232104    2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 07:28:40.243483 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243454    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.243618 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243499    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-conf-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.243618 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243527    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-daemon-config\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.243618 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243545    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-systemd-units\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.243618 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243560    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.243618 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243578    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-ovnkube-config\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.243618 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243600    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtwtf\" (UniqueName: \"kubernetes.io/projected/1a089cc1-9715-4809-9479-02fddca15479-kube-api-access-xtwtf\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.243618 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243622    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-cni-bin\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243638    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243656    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-socket-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243670    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75t9x\" (UniqueName: \"kubernetes.io/projected/0f24a74d-99e3-4bce-a9e1-2eb550655093-kube-api-access-75t9x\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243685    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243698    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243712    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243728    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-kubelet\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243751    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysconfig\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243771    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysctl-d\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243789    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-multus-certs\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243803    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-dbus\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243818    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4s6h\" (UniqueName: \"kubernetes.io/projected/d95920d0-5a98-46bc-9b64-f820f02d5b1d-kube-api-access-n4s6h\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243831    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-hostroot\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243845    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf2415aa-becd-409e-bb51-05d94a380deb-iptables-alerter-script\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243858    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62191e70-8059-4950-b636-b6c58d33b128-host\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:28:40.243876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243873    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-system-cni-dir\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243887    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243906    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-run-netns\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243925    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-systemd\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243938    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-var-lib-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243959    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-run\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.243982    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-cnibin\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244000    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf2415aa-becd-409e-bb51-05d94a380deb-host-slash\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244015    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-kubelet-config\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244031    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-979pp\" (UniqueName: \"kubernetes.io/projected/62191e70-8059-4950-b636-b6c58d33b128-kube-api-access-979pp\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244055    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-etc-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244081    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-run-ovn-kubernetes\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244103    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-env-overrides\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244122    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-modprobe-d\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244141    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-sys\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244161    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/881291b2-1e3f-491d-8dd8-b1500fc5ab24-tmp\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.244384 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244183    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-cni-bin\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244201    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-cni-multus\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244242    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-device-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244263    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbq66\" (UniqueName: \"kubernetes.io/projected/881291b2-1e3f-491d-8dd8-b1500fc5ab24-kube-api-access-kbq66\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244277    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-socket-dir-parent\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244297    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244322    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysctl-conf\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244341    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-tuned\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244364    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-netns\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244386    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-kubelet\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244409    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-hosts-file\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244441    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-tmp-dir\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244467    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-699ww\" (UniqueName: \"kubernetes.io/projected/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-kube-api-access-699ww\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244490    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/62191e70-8059-4950-b636-b6c58d33b128-serviceca\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244518    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vt2p\" (UniqueName: \"kubernetes.io/projected/374fae1c-44be-4729-802f-e190816bf249-kube-api-access-8vt2p\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244544    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e07914e4-a5e8-42b8-889a-5fab59642856-konnectivity-ca\") pod \"konnectivity-agent-6ll6x\" (UID: \"e07914e4-a5e8-42b8-889a-5fab59642856\") " pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:28:40.244907 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244570    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2978v\" (UniqueName: \"kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v\") pod \"network-check-target-pjddt\" (UID: \"72245812-04f1-47b8-a22f-67a61f175b63\") " pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244592    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-ovn\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244619    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-log-socket\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244653    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-cni-netd\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244679    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-ovnkube-script-lib\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244699    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-lib-modules\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244719    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-system-cni-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244743    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-etc-kubernetes\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244759    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-cni-binary-copy\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244774    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-slash\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244788    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1a089cc1-9715-4809-9479-02fddca15479-ovn-node-metrics-cert\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244811    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-registration-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244827    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-etc-selinux\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244855    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-var-lib-kubelet\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244875    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-cni-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244897    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xzg5\" (UniqueName: \"kubernetes.io/projected/ed269b7c-e48c-4732-acc4-c0250a95801f-kube-api-access-4xzg5\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.245451 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244921    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-node-log\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.246040 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244939    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-sys-fs\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.246040 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244954    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-kubernetes\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.246040 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244972    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-systemd\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.246040 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.244985    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-host\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.246040 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.245006    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f24a74d-99e3-4bce-a9e1-2eb550655093-cni-binary-copy\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.246040 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.245026    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-k8s-cni-cncf-io\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.246040 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.245054    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5624d\" (UniqueName: \"kubernetes.io/projected/cf2415aa-becd-409e-bb51-05d94a380deb-kube-api-access-5624d\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:28:40.246040 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.245084    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e07914e4-a5e8-42b8-889a-5fab59642856-agent-certs\") pod \"konnectivity-agent-6ll6x\" (UID: \"e07914e4-a5e8-42b8-889a-5fab59642856\") " pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:28:40.246040 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.245110    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-cnibin\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.281504 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.281469    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 07:28:40.281504 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.281487    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 07:28:40.281504 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.281504    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 07:28:40.281756 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.281619    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 07:28:40.281813 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.281767    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 07:28:40.281813 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.281775    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 07:28:40.281905 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.281808    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 07:28:40.281971 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.281953    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 07:28:40.282091 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282078    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mrcbt\""
Oct 31 07:28:40.282166 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282150    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 07:28:40.282247 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282186    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 07:28:40.282247 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282211    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 07:28:40.282247 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282154    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 07:28:40.282247 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282241    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 07:28:40.282423 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282287    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 07:28:40.282423 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282295    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 07:28:40.282423 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282415    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-qvcsq\""
Oct 31 07:28:40.282559 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282469    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hr94j\""
Oct 31 07:28:40.282559 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282532    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9v74w\""
Oct 31 07:28:40.282637 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282606    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 07:28:40.282669 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282661    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rhfwc\""
Oct 31 07:28:40.282825 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282802    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 07:28:40.282900 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282860    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 07:28:40.282958 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282913    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 07:28:40.282997 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.282962    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 07:28:40.283049 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.283036    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-krpl2\""
Oct 31 07:28:40.283108 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.283094    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 07:28:40.283176 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.283156    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 07:28:40.345331 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345290    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-hostroot\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.345331 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345323    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf2415aa-becd-409e-bb51-05d94a380deb-iptables-alerter-script\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:28:40.345571 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345347    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62191e70-8059-4950-b636-b6c58d33b128-host\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:28:40.345571 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345370    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-system-cni-dir\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.345571 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345395    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.345571 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345414    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-hostroot\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.345571 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345421    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-run-netns\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.345571 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345442    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-systemd\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.345571 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345443    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62191e70-8059-4950-b636-b6c58d33b128-host\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:28:40.345571 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345442    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-system-cni-dir\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.345571 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345479    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-run-netns\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.345571 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345488    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-systemd\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345589    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-var-lib-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345623    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-run\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345648    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-cnibin\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345687    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-var-lib-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345692    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf2415aa-becd-409e-bb51-05d94a380deb-host-slash\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345732    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf2415aa-becd-409e-bb51-05d94a380deb-host-slash\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345736    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-kubelet-config\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345767    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-979pp\" (UniqueName: \"kubernetes.io/projected/62191e70-8059-4950-b636-b6c58d33b128-kube-api-access-979pp\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345788    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-kubelet-config\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345798    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-etc-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345801    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-run\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345833    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-cnibin\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345840    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-etc-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345840    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-run-ovn-kubernetes\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345871    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-env-overrides\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345874    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-run-ovn-kubernetes\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345897    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-modprobe-d\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.345976 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345919    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-sys\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345942    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/881291b2-1e3f-491d-8dd8-b1500fc5ab24-tmp\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345965    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-cni-bin\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.345986    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-cni-multus\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346009    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-device-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346036    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbq66\" (UniqueName: \"kubernetes.io/projected/881291b2-1e3f-491d-8dd8-b1500fc5ab24-kube-api-access-kbq66\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346061    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-socket-dir-parent\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346068    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346058    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf2415aa-becd-409e-bb51-05d94a380deb-iptables-alerter-script\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346126    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-cni-bin\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346137    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-socket-dir-parent\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346181    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-sys\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346264    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-modprobe-d\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346286    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-cni-multus\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346298    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346328    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysctl-conf\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346337    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-env-overrides\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346343    2572 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 07:28:40.346643 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346349    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-device-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346355    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-tuned\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346470    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-netns\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346516    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-netns\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346553    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-kubelet\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346554    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysctl-conf\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346588    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-hosts-file\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346641    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-hosts-file\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346664    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-kubelet\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346669    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-tmp-dir\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346705    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-699ww\" (UniqueName: \"kubernetes.io/projected/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-kube-api-access-699ww\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346728    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/62191e70-8059-4950-b636-b6c58d33b128-serviceca\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346755    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vt2p\" (UniqueName: \"kubernetes.io/projected/374fae1c-44be-4729-802f-e190816bf249-kube-api-access-8vt2p\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346781    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e07914e4-a5e8-42b8-889a-5fab59642856-konnectivity-ca\") pod \"konnectivity-agent-6ll6x\" (UID: \"e07914e4-a5e8-42b8-889a-5fab59642856\") " pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346804    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2978v\" (UniqueName: \"kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v\") pod \"network-check-target-pjddt\" (UID: \"72245812-04f1-47b8-a22f-67a61f175b63\") " pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346827    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-ovn\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346852    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-log-socket\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346863    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.347478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346875    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-cni-netd\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346898    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-ovnkube-script-lib\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346922    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-lib-modules\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346959    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-system-cni-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346966    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-tmp-dir\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.346988    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-etc-kubernetes\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347018    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-cni-binary-copy\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347041    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-slash\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347052    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-cni-netd\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347063    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1a089cc1-9715-4809-9479-02fddca15479-ovn-node-metrics-cert\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347315    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-ovn\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347366    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-log-socket\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347385    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e07914e4-a5e8-42b8-889a-5fab59642856-konnectivity-ca\") pod \"konnectivity-agent-6ll6x\" (UID: \"e07914e4-a5e8-42b8-889a-5fab59642856\") " pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347401    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/62191e70-8059-4950-b636-b6c58d33b128-serviceca\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347409    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-registration-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347437    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-etc-selinux\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347460    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-var-lib-kubelet\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347472    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-registration-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.348330 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347483    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-cni-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347516    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xzg5\" (UniqueName: \"kubernetes.io/projected/ed269b7c-e48c-4732-acc4-c0250a95801f-kube-api-access-4xzg5\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347544    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-node-log\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347582    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-system-cni-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347586    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-sys-fs\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347618    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-kubernetes\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347635    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-sys-fs\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347642    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-systemd\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347545    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-cni-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347666    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-host\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347680    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-etc-kubernetes\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347688    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f24a74d-99e3-4bce-a9e1-2eb550655093-cni-binary-copy\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347695    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-var-lib-kubelet\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347720    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-k8s-cni-cncf-io\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347743    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5624d\" (UniqueName: \"kubernetes.io/projected/cf2415aa-becd-409e-bb51-05d94a380deb-kube-api-access-5624d\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347747    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-kubernetes\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347751    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-slash\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347765    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e07914e4-a5e8-42b8-889a-5fab59642856-agent-certs\") pod \"konnectivity-agent-6ll6x\" (UID: \"e07914e4-a5e8-42b8-889a-5fab59642856\") " pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:28:40.349148 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347791    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-cnibin\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347817    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347828    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-etc-selinux\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347841    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-conf-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347867    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-daemon-config\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347867    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-systemd\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347890    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-systemd-units\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347910    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-host\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347912    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347945    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-ovnkube-config\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347956    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-node-log\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347975    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtwtf\" (UniqueName: \"kubernetes.io/projected/1a089cc1-9715-4809-9479-02fddca15479-kube-api-access-xtwtf\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.347989    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-k8s-cni-cncf-io\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348000    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-cni-bin\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348026    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348031    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-conf-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348013    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.349999 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348054    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-socket-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348057    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-ovnkube-script-lib\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348149    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-lib-modules\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348180    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75t9x\" (UniqueName: \"kubernetes.io/projected/0f24a74d-99e3-4bce-a9e1-2eb550655093-kube-api-access-75t9x\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348195    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-cni-binary-copy\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348210    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348254    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348281    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348308    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-kubelet\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348332    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysconfig\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348355    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysctl-d\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348378    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-multus-certs\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348402    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-dbus\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348434    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4s6h\" (UniqueName: \"kubernetes.io/projected/d95920d0-5a98-46bc-9b64-f820f02d5b1d-kube-api-access-n4s6h\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348458    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-cnibin\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348514    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-kubelet\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348530    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysconfig\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.351988 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348553    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f24a74d-99e3-4bce-a9e1-2eb550655093-cni-binary-copy\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348618    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysctl-d\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348653    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-daemon-config\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348658    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-multus-certs\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348702    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-dbus\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348735    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-systemd-units\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348784    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348805    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-socket-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.348747    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348851    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.348805    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.348885    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret podName:11dc8b0e-286d-42e1-bfe0-d882e7dfe073 nodeName:}" failed. No retries permitted until 2026-10-31 07:28:40.848866508 +0000 UTC m=+3.058201238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret") pod "global-pull-secret-syncer-z9h8d" (UID: "11dc8b0e-286d-42e1-bfe0-d882e7dfe073") : object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.348902    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs podName:d95920d0-5a98-46bc-9b64-f820f02d5b1d nodeName:}" failed. No retries permitted until 2026-10-31 07:28:40.848892536 +0000 UTC m=+3.058227240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs") pod "network-metrics-daemon-2skf7" (UID: "d95920d0-5a98-46bc-9b64-f820f02d5b1d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.348947    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.349016    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-cni-bin\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.349268    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-ovnkube-config\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.350574    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-tuned\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.352826 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.351788    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1a089cc1-9715-4809-9479-02fddca15479-ovn-node-metrics-cert\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.353621 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.352182    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e07914e4-a5e8-42b8-889a-5fab59642856-agent-certs\") pod \"konnectivity-agent-6ll6x\" (UID: \"e07914e4-a5e8-42b8-889a-5fab59642856\") " pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:28:40.353621 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.352330    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/881291b2-1e3f-491d-8dd8-b1500fc5ab24-tmp\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.369498 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.369438    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 07:28:40.369498 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.369464    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 07:28:40.369498 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.369478    2572 projected.go:196] Error preparing data for projected volume kube-api-access-2978v for pod openshift-network-diagnostics/network-check-target-pjddt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:40.369727 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.369556    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v podName:72245812-04f1-47b8-a22f-67a61f175b63 nodeName:}" failed. No retries permitted until 2026-10-31 07:28:40.869536163 +0000 UTC m=+3.078870882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2978v" (UniqueName: "kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v") pod "network-check-target-pjddt" (UID: "72245812-04f1-47b8-a22f-67a61f175b63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:40.372185 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.372099    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-979pp\" (UniqueName: \"kubernetes.io/projected/62191e70-8059-4950-b636-b6c58d33b128-kube-api-access-979pp\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:28:40.372391 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.372368    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vt2p\" (UniqueName: \"kubernetes.io/projected/374fae1c-44be-4729-802f-e190816bf249-kube-api-access-8vt2p\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.373568 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.373521    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-699ww\" (UniqueName: \"kubernetes.io/projected/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-kube-api-access-699ww\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:28:40.374184 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.374159    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5624d\" (UniqueName: \"kubernetes.io/projected/cf2415aa-becd-409e-bb51-05d94a380deb-kube-api-access-5624d\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:28:40.377608 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.375560    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbq66\" (UniqueName: \"kubernetes.io/projected/881291b2-1e3f-491d-8dd8-b1500fc5ab24-kube-api-access-kbq66\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.377608 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.375616    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xzg5\" (UniqueName: \"kubernetes.io/projected/ed269b7c-e48c-4732-acc4-c0250a95801f-kube-api-access-4xzg5\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.377608 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.376051    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4s6h\" (UniqueName: \"kubernetes.io/projected/d95920d0-5a98-46bc-9b64-f820f02d5b1d-kube-api-access-n4s6h\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:40.378053 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.378028    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtwtf\" (UniqueName: \"kubernetes.io/projected/1a089cc1-9715-4809-9479-02fddca15479-kube-api-access-xtwtf\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.378817 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.378789    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75t9x\" (UniqueName: \"kubernetes.io/projected/0f24a74d-99e3-4bce-a9e1-2eb550655093-kube-api-access-75t9x\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.477300 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.477267    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:28:40.488947 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:40.488915    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode07914e4_a5e8_42b8_889a_5fab59642856.slice/crio-8c73287f7d0aa2a7cbd56a488b7d4e9cbd5dfd0549c78c5610c4d4581d8e89e0 WatchSource:0}: Error finding container 8c73287f7d0aa2a7cbd56a488b7d4e9cbd5dfd0549c78c5610c4d4581d8e89e0: Status 404 returned error can't find the container with id 8c73287f7d0aa2a7cbd56a488b7d4e9cbd5dfd0549c78c5610c4d4581d8e89e0
Oct 31 07:28:40.491321 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.491132    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:28:40.496270 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.496178    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:28:40.500962 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:40.500938    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod374fae1c_44be_4729_802f_e190816bf249.slice/crio-9ace0b66dfed7d90a6eafcab8506be3c744bf165e2104be4070aa7e2cf55744a WatchSource:0}: Error finding container 9ace0b66dfed7d90a6eafcab8506be3c744bf165e2104be4070aa7e2cf55744a: Status 404 returned error can't find the container with id 9ace0b66dfed7d90a6eafcab8506be3c744bf165e2104be4070aa7e2cf55744a
Oct 31 07:28:40.502315 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.502270    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jghjq"
Oct 31 07:28:40.506048 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:40.506020    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881291b2_1e3f_491d_8dd8_b1500fc5ab24.slice/crio-b45fbae3ff97c70e866df0af437b6fbe1064dcecf72e796b75db9838dcfd56e5 WatchSource:0}: Error finding container b45fbae3ff97c70e866df0af437b6fbe1064dcecf72e796b75db9838dcfd56e5: Status 404 returned error can't find the container with id b45fbae3ff97c70e866df0af437b6fbe1064dcecf72e796b75db9838dcfd56e5
Oct 31 07:28:40.511073 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.511051    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:28:40.512624 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:40.512598    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f24a74d_99e3_4bce_a9e1_2eb550655093.slice/crio-1306f68e8130e53948cb2eb356fd5d7930dac4bbc36095df328e7375bc5ffdea WatchSource:0}: Error finding container 1306f68e8130e53948cb2eb356fd5d7930dac4bbc36095df328e7375bc5ffdea: Status 404 returned error can't find the container with id 1306f68e8130e53948cb2eb356fd5d7930dac4bbc36095df328e7375bc5ffdea
Oct 31 07:28:40.518597 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.518570    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:28:40.524321 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:40.524292    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf2415aa_becd_409e_bb51_05d94a380deb.slice/crio-daacb703ff4a2581c23b6baa731c6c0428df334169ba8f0b6a522387e75f9e71 WatchSource:0}: Error finding container daacb703ff4a2581c23b6baa731c6c0428df334169ba8f0b6a522387e75f9e71: Status 404 returned error can't find the container with id daacb703ff4a2581c23b6baa731c6c0428df334169ba8f0b6a522387e75f9e71
Oct 31 07:28:40.527425 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.527219    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:28:40.529965 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:40.529928    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62191e70_8059_4950_b636_b6c58d33b128.slice/crio-83315d191c3d017dcefd0d2fe3ee667ae8e2d9d62dacc42a89fedbb045e39319 WatchSource:0}: Error finding container 83315d191c3d017dcefd0d2fe3ee667ae8e2d9d62dacc42a89fedbb045e39319: Status 404 returned error can't find the container with id 83315d191c3d017dcefd0d2fe3ee667ae8e2d9d62dacc42a89fedbb045e39319
Oct 31 07:28:40.531757 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.531740    2572 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:28:40.539390 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:40.539359    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ca5663_7cde_4b55_b971_e2afaf82fb5f.slice/crio-828e47b40abf587bf9f22ade36b0923d9911ce9d6a1641a9af54fe8f7ae953f8 WatchSource:0}: Error finding container 828e47b40abf587bf9f22ade36b0923d9911ce9d6a1641a9af54fe8f7ae953f8: Status 404 returned error can't find the container with id 828e47b40abf587bf9f22ade36b0923d9911ce9d6a1641a9af54fe8f7ae953f8
Oct 31 07:28:40.554358 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.554317    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:28:40.563917 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:40.563886    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded269b7c_e48c_4732_acc4_c0250a95801f.slice/crio-35807caf7071ae80511bf71e7553e3f12001be5ddf3f0ae1c51c87e5fbacbb16 WatchSource:0}: Error finding container 35807caf7071ae80511bf71e7553e3f12001be5ddf3f0ae1c51c87e5fbacbb16: Status 404 returned error can't find the container with id 35807caf7071ae80511bf71e7553e3f12001be5ddf3f0ae1c51c87e5fbacbb16
Oct 31 07:28:40.566737 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.566715    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:40.575926 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:28:40.575897    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a089cc1_9715_4809_9479_02fddca15479.slice/crio-6880eb0e8fe90c744e2d7d2097a3361b4478005f3e29885f235f64bdc4ca5303 WatchSource:0}: Error finding container 6880eb0e8fe90c744e2d7d2097a3361b4478005f3e29885f235f64bdc4ca5303: Status 404 returned error can't find the container with id 6880eb0e8fe90c744e2d7d2097a3361b4478005f3e29885f235f64bdc4ca5303
Oct 31 07:28:40.852479 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.852239    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:40.852602 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.852492    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:40.852658 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.852635    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:40.852723 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.852681    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs podName:d95920d0-5a98-46bc-9b64-f820f02d5b1d nodeName:}" failed. No retries permitted until 2026-10-31 07:28:41.852669419 +0000 UTC m=+4.062004122 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs") pod "network-metrics-daemon-2skf7" (UID: "d95920d0-5a98-46bc-9b64-f820f02d5b1d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:40.852786 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.852729    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:40.852786 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.852762    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret podName:11dc8b0e-286d-42e1-bfe0-d882e7dfe073 nodeName:}" failed. No retries permitted until 2026-10-31 07:28:41.852752641 +0000 UTC m=+4.062087359 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret") pod "global-pull-secret-syncer-z9h8d" (UID: "11dc8b0e-286d-42e1-bfe0-d882e7dfe073") : object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:40.953640 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:40.952921    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2978v\" (UniqueName: \"kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v\") pod \"network-check-target-pjddt\" (UID: \"72245812-04f1-47b8-a22f-67a61f175b63\") " pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:40.953640 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.953131    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 07:28:40.953640 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.953153    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 07:28:40.953640 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.953165    2572 projected.go:196] Error preparing data for projected volume kube-api-access-2978v for pod openshift-network-diagnostics/network-check-target-pjddt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:40.953640 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:40.953221    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v podName:72245812-04f1-47b8-a22f-67a61f175b63 nodeName:}" failed. No retries permitted until 2026-10-31 07:28:41.953203771 +0000 UTC m=+4.162538481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2978v" (UniqueName: "kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v") pod "network-check-target-pjddt" (UID: "72245812-04f1-47b8-a22f-67a61f175b63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:41.196385 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.196316    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 07:23:40 +0000 UTC" deadline="2028-06-06 19:13:04.275554519 +0000 UTC"
Oct 31 07:28:41.196385 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.196380    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14027h44m23.079178835s"
Oct 31 07:28:41.375379 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.375345    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:41.375542 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:41.375479    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjddt" podUID="72245812-04f1-47b8-a22f-67a61f175b63"
Oct 31 07:28:41.396161 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.396079    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fl4w7" event={"ID":"62191e70-8059-4950-b636-b6c58d33b128","Type":"ContainerStarted","Data":"83315d191c3d017dcefd0d2fe3ee667ae8e2d9d62dacc42a89fedbb045e39319"}
Oct 31 07:28:41.407253 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.407158    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z22fp" event={"ID":"cf2415aa-becd-409e-bb51-05d94a380deb","Type":"ContainerStarted","Data":"daacb703ff4a2581c23b6baa731c6c0428df334169ba8f0b6a522387e75f9e71"}
Oct 31 07:28:41.409539 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.409466    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m7rmk" event={"ID":"881291b2-1e3f-491d-8dd8-b1500fc5ab24","Type":"ContainerStarted","Data":"b45fbae3ff97c70e866df0af437b6fbe1064dcecf72e796b75db9838dcfd56e5"}
Oct 31 07:28:41.418176 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.417406    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal" event={"ID":"700ea6542926bcf6606c986e3906f152","Type":"ContainerStarted","Data":"6f1dfe8c0ef1b1d2af64e7e7709829070dc213e5ed7b3f10e931c4c13b31fc5a"}
Oct 31 07:28:41.437485 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.437320    2572 generic.go:356] "Generic (PLEG): container finished" podID="ee833da619c66751ed2de8e982aff1e5" containerID="45347de82f09b36205869e5d68561611d0a44af1823936038adeaedcbb4121fd" exitCode=0
Oct 31 07:28:41.437485 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.437420    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" event={"ID":"ee833da619c66751ed2de8e982aff1e5","Type":"ContainerDied","Data":"45347de82f09b36205869e5d68561611d0a44af1823936038adeaedcbb4121fd"}
Oct 31 07:28:41.446430 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.446312    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4qfb" event={"ID":"ed269b7c-e48c-4732-acc4-c0250a95801f","Type":"ContainerStarted","Data":"35807caf7071ae80511bf71e7553e3f12001be5ddf3f0ae1c51c87e5fbacbb16"}
Oct 31 07:28:41.458265 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.458180    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jghjq" event={"ID":"0f24a74d-99e3-4bce-a9e1-2eb550655093","Type":"ContainerStarted","Data":"1306f68e8130e53948cb2eb356fd5d7930dac4bbc36095df328e7375bc5ffdea"}
Oct 31 07:28:41.462773 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.462425    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal" podStartSLOduration=2.46240822 podStartE2EDuration="2.46240822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:41.461336311 +0000 UTC m=+3.670671038" watchObservedRunningTime="2026-10-31 07:28:41.46240822 +0000 UTC m=+3.671742947"
Oct 31 07:28:41.464739 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.464649    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l" event={"ID":"374fae1c-44be-4729-802f-e190816bf249","Type":"ContainerStarted","Data":"9ace0b66dfed7d90a6eafcab8506be3c744bf165e2104be4070aa7e2cf55744a"}
Oct 31 07:28:41.475083 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.475005    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6ll6x" event={"ID":"e07914e4-a5e8-42b8-889a-5fab59642856","Type":"ContainerStarted","Data":"8c73287f7d0aa2a7cbd56a488b7d4e9cbd5dfd0549c78c5610c4d4581d8e89e0"}
Oct 31 07:28:41.498825 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.498693    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4" event={"ID":"1a089cc1-9715-4809-9479-02fddca15479","Type":"ContainerStarted","Data":"6880eb0e8fe90c744e2d7d2097a3361b4478005f3e29885f235f64bdc4ca5303"}
Oct 31 07:28:41.515337 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.515248    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gkl8q" event={"ID":"b1ca5663-7cde-4b55-b971-e2afaf82fb5f","Type":"ContainerStarted","Data":"828e47b40abf587bf9f22ade36b0923d9911ce9d6a1641a9af54fe8f7ae953f8"}
Oct 31 07:28:41.861668 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.861631    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:41.861834 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.861678    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:41.861834 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:41.861821    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:41.861951 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:41.861884    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs podName:d95920d0-5a98-46bc-9b64-f820f02d5b1d nodeName:}" failed. No retries permitted until 2026-10-31 07:28:43.861866442 +0000 UTC m=+6.071201148 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs") pod "network-metrics-daemon-2skf7" (UID: "d95920d0-5a98-46bc-9b64-f820f02d5b1d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:41.871423 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:41.871391    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:41.871587 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:41.871481    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret podName:11dc8b0e-286d-42e1-bfe0-d882e7dfe073 nodeName:}" failed. No retries permitted until 2026-10-31 07:28:43.871460172 +0000 UTC m=+6.080794881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret") pod "global-pull-secret-syncer-z9h8d" (UID: "11dc8b0e-286d-42e1-bfe0-d882e7dfe073") : object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:41.963034 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:41.962944    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2978v\" (UniqueName: \"kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v\") pod \"network-check-target-pjddt\" (UID: \"72245812-04f1-47b8-a22f-67a61f175b63\") " pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:41.963197 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:41.963163    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 07:28:41.963197 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:41.963185    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 07:28:41.963197 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:41.963198    2572 projected.go:196] Error preparing data for projected volume kube-api-access-2978v for pod openshift-network-diagnostics/network-check-target-pjddt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:41.963365 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:41.963289    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v podName:72245812-04f1-47b8-a22f-67a61f175b63 nodeName:}" failed. No retries permitted until 2026-10-31 07:28:43.963270796 +0000 UTC m=+6.172605518 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2978v" (UniqueName: "kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v") pod "network-check-target-pjddt" (UID: "72245812-04f1-47b8-a22f-67a61f175b63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:42.197218 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:42.197125    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 07:23:40 +0000 UTC" deadline="2028-04-10 11:55:48.835439003 +0000 UTC"
Oct 31 07:28:42.197218 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:42.197187    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12652h27m6.638256272s"
Oct 31 07:28:42.377703 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:42.377666    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:42.377889 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:42.377792    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h8d" podUID="11dc8b0e-286d-42e1-bfe0-d882e7dfe073"
Oct 31 07:28:42.378186 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:42.378161    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:42.378307 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:42.378288    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2skf7" podUID="d95920d0-5a98-46bc-9b64-f820f02d5b1d"
Oct 31 07:28:42.534593 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:42.534304    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" event={"ID":"ee833da619c66751ed2de8e982aff1e5","Type":"ContainerStarted","Data":"bdc45b5dd4a27256b57033750a80a9d6411e41d9ccc94366d1927e6cd02406ed"}
Oct 31 07:28:43.375740 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:43.375072    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:43.375740 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:43.375187    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjddt" podUID="72245812-04f1-47b8-a22f-67a61f175b63"
Oct 31 07:28:43.540397 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:43.539128    2572 generic.go:356] "Generic (PLEG): container finished" podID="ed269b7c-e48c-4732-acc4-c0250a95801f" containerID="0b435dd8d7cd695a80ae77537cf8c506530e93ccebe8e12a1cf482bb18c50bc1" exitCode=0
Oct 31 07:28:43.540397 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:43.540166    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4qfb" event={"ID":"ed269b7c-e48c-4732-acc4-c0250a95801f","Type":"ContainerDied","Data":"0b435dd8d7cd695a80ae77537cf8c506530e93ccebe8e12a1cf482bb18c50bc1"}
Oct 31 07:28:43.590651 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:43.590591    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" podStartSLOduration=4.590574794 podStartE2EDuration="4.590574794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:42.578878302 +0000 UTC m=+4.788213029" watchObservedRunningTime="2026-10-31 07:28:43.590574794 +0000 UTC m=+5.799909520"
Oct 31 07:28:43.880421 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:43.880385    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:43.880603 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:43.880433    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:43.880663 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:43.880603    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:43.880717 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:43.880667    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs podName:d95920d0-5a98-46bc-9b64-f820f02d5b1d nodeName:}" failed. No retries permitted until 2026-10-31 07:28:47.880649101 +0000 UTC m=+10.089983806 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs") pod "network-metrics-daemon-2skf7" (UID: "d95920d0-5a98-46bc-9b64-f820f02d5b1d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:43.880775 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:43.880738    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:43.880775 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:43.880770    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret podName:11dc8b0e-286d-42e1-bfe0-d882e7dfe073 nodeName:}" failed. No retries permitted until 2026-10-31 07:28:47.880759414 +0000 UTC m=+10.090094132 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret") pod "global-pull-secret-syncer-z9h8d" (UID: "11dc8b0e-286d-42e1-bfe0-d882e7dfe073") : object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:43.981939 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:43.981410    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2978v\" (UniqueName: \"kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v\") pod \"network-check-target-pjddt\" (UID: \"72245812-04f1-47b8-a22f-67a61f175b63\") " pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:43.981939 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:43.981542    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 07:28:43.981939 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:43.981556    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 07:28:43.981939 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:43.981565    2572 projected.go:196] Error preparing data for projected volume kube-api-access-2978v for pod openshift-network-diagnostics/network-check-target-pjddt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:43.981939 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:43.981610    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v podName:72245812-04f1-47b8-a22f-67a61f175b63 nodeName:}" failed. No retries permitted until 2026-10-31 07:28:47.98159947 +0000 UTC m=+10.190934187 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2978v" (UniqueName: "kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v") pod "network-check-target-pjddt" (UID: "72245812-04f1-47b8-a22f-67a61f175b63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:44.378304 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:44.378168    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:44.378693 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:44.378305    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h8d" podUID="11dc8b0e-286d-42e1-bfe0-d882e7dfe073"
Oct 31 07:28:44.378693 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:44.378672    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:44.378813 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:44.378773    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2skf7" podUID="d95920d0-5a98-46bc-9b64-f820f02d5b1d"
Oct 31 07:28:44.546526 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:44.546492    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6ll6x" event={"ID":"e07914e4-a5e8-42b8-889a-5fab59642856","Type":"ContainerStarted","Data":"6996d8d4cbd5cfdf2ec22718a665c52b3ce10a53df9d7b988c04058c3a381dbe"}
Oct 31 07:28:45.377480 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:45.377449    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:45.377643 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:45.377617    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjddt" podUID="72245812-04f1-47b8-a22f-67a61f175b63"
Oct 31 07:28:46.233191 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:46.233066    2572 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:28:46.233961 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:46.233823    2572 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:28:46.280691 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:46.280645    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6ll6x" podStartSLOduration=4.526993521 podStartE2EDuration="8.280631735s" totalImagesPullingTime="3.753638214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:44.590923131 +0000 UTC m=+6.800257857" watchObservedRunningTime="2026-10-31 07:28:46.280631735 +0000 UTC m=+8.489966461"
Oct 31 07:28:46.376994 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:46.376959    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:46.377148 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:46.377077    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h8d" podUID="11dc8b0e-286d-42e1-bfe0-d882e7dfe073"
Oct 31 07:28:46.377674 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:46.377469    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:46.377674 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:46.377575    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2skf7" podUID="d95920d0-5a98-46bc-9b64-f820f02d5b1d"
Oct 31 07:28:46.556911 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:46.556331    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fl4w7" event={"ID":"62191e70-8059-4950-b636-b6c58d33b128","Type":"ContainerStarted","Data":"8350ac0dbabc7730eb8bcb6d70cffd930d93a161f0b6cb13d9ccb30feab7c1f8"}
Oct 31 07:28:46.559982 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:46.559901    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l" event={"ID":"374fae1c-44be-4729-802f-e190816bf249","Type":"ContainerStarted","Data":"c7aea3c81909b54d970baf5138915ce8e8660bcfea17f64814ceccdae3154396"}
Oct 31 07:28:47.374921 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:47.374889    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:47.375456 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:47.375014    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjddt" podUID="72245812-04f1-47b8-a22f-67a61f175b63"
Oct 31 07:28:47.561733 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:47.561702    2572 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 07:28:47.786805 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:47.786757    2572 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 07:28:47.795431 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:47.795329    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:28:47.797209 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:47.797035    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:28:47.842955 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:47.842457    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fl4w7" podStartSLOduration=5.00596478 podStartE2EDuration="9.842438632s" totalImagesPullingTime="4.836473852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:46.600644435 +0000 UTC m=+8.809979162" watchObservedRunningTime="2026-10-31 07:28:47.842438632 +0000 UTC m=+10.051773360"
Oct 31 07:28:47.916398 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:47.916319    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:47.916398 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:47.916377    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:47.916597 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:47.916578    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:47.916664 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:47.916647    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs podName:d95920d0-5a98-46bc-9b64-f820f02d5b1d nodeName:}" failed. No retries permitted until 2026-10-31 07:28:55.916628425 +0000 UTC m=+18.125963144 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs") pod "network-metrics-daemon-2skf7" (UID: "d95920d0-5a98-46bc-9b64-f820f02d5b1d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:47.917077 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:47.917057    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:47.917160 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:47.917122    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret podName:11dc8b0e-286d-42e1-bfe0-d882e7dfe073 nodeName:}" failed. No retries permitted until 2026-10-31 07:28:55.917101804 +0000 UTC m=+18.126436523 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret") pod "global-pull-secret-syncer-z9h8d" (UID: "11dc8b0e-286d-42e1-bfe0-d882e7dfe073") : object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:48.017711 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:48.017664    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2978v\" (UniqueName: \"kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v\") pod \"network-check-target-pjddt\" (UID: \"72245812-04f1-47b8-a22f-67a61f175b63\") " pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:48.017906 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:48.017886    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 07:28:48.017993 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:48.017913    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 07:28:48.017993 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:48.017926    2572 projected.go:196] Error preparing data for projected volume kube-api-access-2978v for pod openshift-network-diagnostics/network-check-target-pjddt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:48.018087 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:48.017997    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v podName:72245812-04f1-47b8-a22f-67a61f175b63 nodeName:}" failed. No retries permitted until 2026-10-31 07:28:56.017977767 +0000 UTC m=+18.227312475 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2978v" (UniqueName: "kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v") pod "network-check-target-pjddt" (UID: "72245812-04f1-47b8-a22f-67a61f175b63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:48.201507 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:48.194411    2572 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T07:28:47.78677673Z","UUID":"e8ec1133-40c9-4c1e-a62a-4cc679b7ebcf","Handler":null,"Name":"","Endpoint":""}
Oct 31 07:28:48.205133 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:48.205116    2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 07:28:48.205280 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:48.205143    2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 07:28:48.376512 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:48.376254    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:48.376875 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:48.376299    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:48.376875 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:48.376623    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h8d" podUID="11dc8b0e-286d-42e1-bfe0-d882e7dfe073"
Oct 31 07:28:48.376875 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:48.376713    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2skf7" podUID="d95920d0-5a98-46bc-9b64-f820f02d5b1d"
Oct 31 07:28:48.567994 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:48.567963    2572 generic.go:356] "Generic (PLEG): container finished" podID="ed269b7c-e48c-4732-acc4-c0250a95801f" containerID="3afe7ff113255d0cda263f4c817a20ece5475cb12253dd90bf505a2fd2de6446" exitCode=0
Oct 31 07:28:48.568361 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:48.568296    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4qfb" event={"ID":"ed269b7c-e48c-4732-acc4-c0250a95801f","Type":"ContainerDied","Data":"3afe7ff113255d0cda263f4c817a20ece5475cb12253dd90bf505a2fd2de6446"}
Oct 31 07:28:48.569933 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:48.569904    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jghjq" event={"ID":"0f24a74d-99e3-4bce-a9e1-2eb550655093","Type":"ContainerStarted","Data":"0f443ce1e070a39a582c47bc05ee685a0b64b81ffa40686b12f97500c01d61de"}
Oct 31 07:28:48.572613 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:48.572589    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l" event={"ID":"374fae1c-44be-4729-802f-e190816bf249","Type":"ContainerStarted","Data":"da761229c6485fd2f9b21cfb194856fd6d5b14b165a696db7c63dc3d6bfb6be7"}
Oct 31 07:28:48.574598 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:48.574572    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gkl8q" event={"ID":"b1ca5663-7cde-4b55-b971-e2afaf82fb5f","Type":"ContainerStarted","Data":"f0e653713c99e8a2143b288fcec60baeb8f840b9f14fc8142ae7f915dd02ce83"}
Oct 31 07:28:48.663682 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:48.663626    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-gkl8q" podStartSLOduration=3.281383486 podStartE2EDuration="10.663608373s" totalImagesPullingTime="7.382224887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:48.662787145 +0000 UTC m=+10.872121872" watchObservedRunningTime="2026-10-31 07:28:48.663608373 +0000 UTC m=+10.872943099"
Oct 31 07:28:49.375895 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:49.375865    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:49.376049 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:49.375984    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjddt" podUID="72245812-04f1-47b8-a22f-67a61f175b63"
Oct 31 07:28:49.579718 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:49.579678    2572 generic.go:356] "Generic (PLEG): container finished" podID="ed269b7c-e48c-4732-acc4-c0250a95801f" containerID="ce18a6398904b5dcfb31552c6187ba05339d337ae60a062d24ce0befdb778b0b" exitCode=0
Oct 31 07:28:49.580264 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:49.579759    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4qfb" event={"ID":"ed269b7c-e48c-4732-acc4-c0250a95801f","Type":"ContainerDied","Data":"ce18a6398904b5dcfb31552c6187ba05339d337ae60a062d24ce0befdb778b0b"}
Oct 31 07:28:49.582722 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:49.582671    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l" event={"ID":"374fae1c-44be-4729-802f-e190816bf249","Type":"ContainerStarted","Data":"bba99cec152180681c5681861f76af0b9c8ebf492ffcc86cb71cb5abf0bbf347"}
Oct 31 07:28:49.630246 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:49.630175    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-jghjq" podStartSLOduration=3.91476709 podStartE2EDuration="11.630162138s" totalImagesPullingTime="7.715395048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:48.709934367 +0000 UTC m=+10.919269097" watchObservedRunningTime="2026-10-31 07:28:49.630162138 +0000 UTC m=+11.839496864"
Oct 31 07:28:49.676850 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:49.676792    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l" podStartSLOduration=3.806299494 podStartE2EDuration="11.676772349s" totalImagesPullingTime="7.870472855s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:49.675984372 +0000 UTC m=+11.885319099" watchObservedRunningTime="2026-10-31 07:28:49.676772349 +0000 UTC m=+11.886107076"
Oct 31 07:28:50.376442 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:50.375929    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:50.376442 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:50.376075    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2skf7" podUID="d95920d0-5a98-46bc-9b64-f820f02d5b1d"
Oct 31 07:28:50.376442 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:50.376144    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:50.376442 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:50.376295    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h8d" podUID="11dc8b0e-286d-42e1-bfe0-d882e7dfe073"
Oct 31 07:28:50.587337 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:50.586848    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z22fp" event={"ID":"cf2415aa-becd-409e-bb51-05d94a380deb","Type":"ContainerStarted","Data":"6d27204912d9c2e953389b7e0ffed655478871738340cdb7df45f83a0ae989d8"}
Oct 31 07:28:50.630723 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:50.630655    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z22fp" podStartSLOduration=4.790019727 podStartE2EDuration="12.63063974s" totalImagesPullingTime="7.840620013s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:50.629959505 +0000 UTC m=+12.839294231" watchObservedRunningTime="2026-10-31 07:28:50.63063974 +0000 UTC m=+12.839974465"
Oct 31 07:28:51.375293 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:51.375261    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:51.375478 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:51.375377    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjddt" podUID="72245812-04f1-47b8-a22f-67a61f175b63"
Oct 31 07:28:51.591062 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:51.591024    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m7rmk" event={"ID":"881291b2-1e3f-491d-8dd8-b1500fc5ab24","Type":"ContainerStarted","Data":"8999c5f5a2248ca7aebcafcc3bbd6856ae9b1d09c3d61831266d6cc05381bc10"}
Oct 31 07:28:51.594114 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:51.594089    2572 generic.go:356] "Generic (PLEG): container finished" podID="ed269b7c-e48c-4732-acc4-c0250a95801f" containerID="62aae0bcd549349da519b346b911df76183d3d297fa3ba7f61576ba2fb80c903" exitCode=0
Oct 31 07:28:51.594261 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:51.594164    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4qfb" event={"ID":"ed269b7c-e48c-4732-acc4-c0250a95801f","Type":"ContainerDied","Data":"62aae0bcd549349da519b346b911df76183d3d297fa3ba7f61576ba2fb80c903"}
Oct 31 07:28:51.637042 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:51.636937    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m7rmk" podStartSLOduration=3.634830734 podStartE2EDuration="13.636921473s" totalImagesPullingTime="10.002090739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:51.636534145 +0000 UTC m=+13.845868873" watchObservedRunningTime="2026-10-31 07:28:51.636921473 +0000 UTC m=+13.846256202"
Oct 31 07:28:52.379117 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:52.379082    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:52.379304 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:52.379204    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2skf7" podUID="d95920d0-5a98-46bc-9b64-f820f02d5b1d"
Oct 31 07:28:52.379564 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:52.379548    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:52.379651 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:52.379636    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h8d" podUID="11dc8b0e-286d-42e1-bfe0-d882e7dfe073"
Oct 31 07:28:53.375052 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:53.374848    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:53.375481 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:53.375140    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjddt" podUID="72245812-04f1-47b8-a22f-67a61f175b63"
Oct 31 07:28:53.601426 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:53.601318    2572 generic.go:356] "Generic (PLEG): container finished" podID="ed269b7c-e48c-4732-acc4-c0250a95801f" containerID="d97137795423119cad0214038fa74f2bcb1c1bb2a7804471a045f6c74defafa6" exitCode=0
Oct 31 07:28:53.601426 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:53.601356    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4qfb" event={"ID":"ed269b7c-e48c-4732-acc4-c0250a95801f","Type":"ContainerDied","Data":"d97137795423119cad0214038fa74f2bcb1c1bb2a7804471a045f6c74defafa6"}
Oct 31 07:28:54.378169 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:54.378139    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:54.378631 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:54.378279    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2skf7" podUID="d95920d0-5a98-46bc-9b64-f820f02d5b1d"
Oct 31 07:28:54.378631 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:54.378318    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:54.378631 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:54.378396    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h8d" podUID="11dc8b0e-286d-42e1-bfe0-d882e7dfe073"
Oct 31 07:28:54.608849 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:54.608820    2572 generic.go:356] "Generic (PLEG): container finished" podID="ed269b7c-e48c-4732-acc4-c0250a95801f" containerID="c7ba45d3b9a3444f3cef07e45b0da9950050f0d007875035296e1f590fecf11c" exitCode=0
Oct 31 07:28:54.609009 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:54.608866    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4qfb" event={"ID":"ed269b7c-e48c-4732-acc4-c0250a95801f","Type":"ContainerDied","Data":"c7ba45d3b9a3444f3cef07e45b0da9950050f0d007875035296e1f590fecf11c"}
Oct 31 07:28:55.375909 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:55.375684    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:55.376030 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:55.376005    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjddt" podUID="72245812-04f1-47b8-a22f-67a61f175b63"
Oct 31 07:28:55.613151 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:55.613069    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4" event={"ID":"1a089cc1-9715-4809-9479-02fddca15479","Type":"ContainerStarted","Data":"f9040aca82db8f49eb16733e79546c461b81d751ea1fe6a2b5d1ab8a2c556ed3"}
Oct 31 07:28:55.613151 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:55.613134    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4" event={"ID":"1a089cc1-9715-4809-9479-02fddca15479","Type":"ContainerStarted","Data":"e38fcf04370257e87dfa8c9e85b813d0a70f455cd5a5f4bf9f2591fb4945d2e5"}
Oct 31 07:28:55.613688 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:55.613176    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4" event={"ID":"1a089cc1-9715-4809-9479-02fddca15479","Type":"ContainerStarted","Data":"6b29b430942d086f41525cb672227bfe9a69a340c970411171d108edda83bebc"}
Oct 31 07:28:55.613688 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:55.613207    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4" event={"ID":"1a089cc1-9715-4809-9479-02fddca15479","Type":"ContainerStarted","Data":"3671fabeb99bd053c5d136cd090192dea89533482eb1d25dd30ebc88cbaa1244"}
Oct 31 07:28:55.613688 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:55.613274    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4" event={"ID":"1a089cc1-9715-4809-9479-02fddca15479","Type":"ContainerStarted","Data":"cbfa20f08698f85d6a1329bc46872aff225a02ade819594974f1c85e64968349"}
Oct 31 07:28:55.613688 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:55.613306    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4" event={"ID":"1a089cc1-9715-4809-9479-02fddca15479","Type":"ContainerStarted","Data":"4dd039a6cda4370bfc005d6ed548a59d20b9e66b9fb9ea86c2f720abf4cc24c7"}
Oct 31 07:28:55.616025 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:55.615998    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h4qfb" event={"ID":"ed269b7c-e48c-4732-acc4-c0250a95801f","Type":"ContainerStarted","Data":"94002473fb1cdfb02fb7f4e08b89e56da047996ba4c044dbab916a1ffde5e6f9"}
Oct 31 07:28:55.668616 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:55.668568    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-h4qfb" podStartSLOduration=7.491170993 podStartE2EDuration="17.668553713s" totalImagesPullingTime="10.17738272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:55.668331073 +0000 UTC m=+17.877665801" watchObservedRunningTime="2026-10-31 07:28:55.668553713 +0000 UTC m=+17.877888447"
Oct 31 07:28:55.979340 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:55.979249    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:55.979340 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:55.979284    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:55.979532 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:55.979398    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:55.979532 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:55.979466    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret podName:11dc8b0e-286d-42e1-bfe0-d882e7dfe073 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:11.979449839 +0000 UTC m=+34.188784552 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret") pod "global-pull-secret-syncer-z9h8d" (UID: "11dc8b0e-286d-42e1-bfe0-d882e7dfe073") : object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:55.979532 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:55.979487    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:55.979532 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:55.979523    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs podName:d95920d0-5a98-46bc-9b64-f820f02d5b1d nodeName:}" failed. No retries permitted until 2026-10-31 07:29:11.979513082 +0000 UTC m=+34.188847800 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs") pod "network-metrics-daemon-2skf7" (UID: "d95920d0-5a98-46bc-9b64-f820f02d5b1d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:56.079807 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:56.079773    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2978v\" (UniqueName: \"kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v\") pod \"network-check-target-pjddt\" (UID: \"72245812-04f1-47b8-a22f-67a61f175b63\") " pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:56.079987 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:56.079903    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 07:28:56.079987 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:56.079920    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 07:28:56.079987 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:56.079933    2572 projected.go:196] Error preparing data for projected volume kube-api-access-2978v for pod openshift-network-diagnostics/network-check-target-pjddt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:56.079987 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:56.079979    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v podName:72245812-04f1-47b8-a22f-67a61f175b63 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:12.079967939 +0000 UTC m=+34.289302643 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2978v" (UniqueName: "kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v") pod "network-check-target-pjddt" (UID: "72245812-04f1-47b8-a22f-67a61f175b63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:56.375656 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:56.375630    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:56.375656 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:56.375663    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:56.378589 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:56.376077    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2skf7" podUID="d95920d0-5a98-46bc-9b64-f820f02d5b1d"
Oct 31 07:28:56.378589 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:56.376219    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h8d" podUID="11dc8b0e-286d-42e1-bfe0-d882e7dfe073"
Oct 31 07:28:57.375334 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:57.375298    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:57.375711 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:57.375391    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjddt" podUID="72245812-04f1-47b8-a22f-67a61f175b63"
Oct 31 07:28:57.623826 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:57.623786    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4" event={"ID":"1a089cc1-9715-4809-9479-02fddca15479","Type":"ContainerStarted","Data":"d23619ef1343916855f4165de4ac651bc4215f758b49c96d33a56e29dae9b4dc"}
Oct 31 07:28:58.376120 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:58.376090    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:28:58.376605 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:58.376215    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:28:58.376605 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:58.376215    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2skf7" podUID="d95920d0-5a98-46bc-9b64-f820f02d5b1d"
Oct 31 07:28:58.376605 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:58.376299    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h8d" podUID="11dc8b0e-286d-42e1-bfe0-d882e7dfe073"
Oct 31 07:28:59.375820 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:59.375791    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:28:59.375986 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:28:59.375898    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjddt" podUID="72245812-04f1-47b8-a22f-67a61f175b63"
Oct 31 07:28:59.632292 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:59.632205    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4" event={"ID":"1a089cc1-9715-4809-9479-02fddca15479","Type":"ContainerStarted","Data":"aa1688ae6f100e9a576bd4b8f0b46ae35a2492498cdbda5dcdd887957ae7d4b2"}
Oct 31 07:28:59.632872 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:59.632567    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:59.632872 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:59.632634    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:59.649354 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:59.649076    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:59.649636 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:59.649621    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:28:59.732450 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:28:59.732403    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4" podStartSLOduration=7.250604188 podStartE2EDuration="21.732389836s" totalImagesPullingTime="14.481785648s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:59.731880286 +0000 UTC m=+21.941215012" watchObservedRunningTime="2026-10-31 07:28:59.732389836 +0000 UTC m=+21.941724563"
Oct 31 07:29:00.375347 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:00.375318    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:29:00.375478 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:00.375318    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:00.375478 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:00.375432    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2skf7" podUID="d95920d0-5a98-46bc-9b64-f820f02d5b1d"
Oct 31 07:29:00.375575 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:00.375479    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h8d" podUID="11dc8b0e-286d-42e1-bfe0-d882e7dfe073"
Oct 31 07:29:00.633983 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:00.633917    2572 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 07:29:01.375594 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:01.375568    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:29:01.375728 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:01.375669    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjddt" podUID="72245812-04f1-47b8-a22f-67a61f175b63"
Oct 31 07:29:01.442244 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:01.442198    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:01.596178 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:01.596100    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pjddt"]
Oct 31 07:29:01.599155 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:01.599129    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2skf7"]
Oct 31 07:29:01.599435 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:01.599414    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:29:01.599571 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:01.599548    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2skf7" podUID="d95920d0-5a98-46bc-9b64-f820f02d5b1d"
Oct 31 07:29:01.610769 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:01.610745    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z9h8d"]
Oct 31 07:29:01.610860 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:01.610824    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:01.610914 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:01.610897    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h8d" podUID="11dc8b0e-286d-42e1-bfe0-d882e7dfe073"
Oct 31 07:29:01.638952 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:01.638925    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:29:01.639351 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:01.639051    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjddt" podUID="72245812-04f1-47b8-a22f-67a61f175b63"
Oct 31 07:29:03.375238 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:03.375206    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:29:03.375701 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:03.375344    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:03.375701 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:03.375360    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjddt" podUID="72245812-04f1-47b8-a22f-67a61f175b63"
Oct 31 07:29:03.375701 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:03.375410    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h8d" podUID="11dc8b0e-286d-42e1-bfe0-d882e7dfe073"
Oct 31 07:29:03.375701 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:03.375470    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:29:03.375701 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:03.375550    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2skf7" podUID="d95920d0-5a98-46bc-9b64-f820f02d5b1d"
Oct 31 07:29:05.375092 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:05.375062    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:29:05.375648 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:05.375166    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:05.375648 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:05.375193    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:29:05.375648 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:05.375185    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pjddt" podUID="72245812-04f1-47b8-a22f-67a61f175b63"
Oct 31 07:29:05.375648 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:05.375277    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9h8d" podUID="11dc8b0e-286d-42e1-bfe0-d882e7dfe073"
Oct 31 07:29:05.375648 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:05.375344    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2skf7" podUID="d95920d0-5a98-46bc-9b64-f820f02d5b1d"
Oct 31 07:29:07.165922 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.165849    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeReady"
Oct 31 07:29:07.166272 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.165968    2572 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 31 07:29:07.228649 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.228610    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vktw2"]
Oct 31 07:29:07.231540 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.231524    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vktw2"
Oct 31 07:29:07.243581 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.243561    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-w78n9"]
Oct 31 07:29:07.246221 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.246207    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:07.249003 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.248989    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 07:29:07.249100 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.249082    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 07:29:07.249145 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.249087    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-cp4x6\""
Oct 31 07:29:07.251941 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.251924    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 07:29:07.258301 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.258280    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vktw2"]
Oct 31 07:29:07.268647 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.268629    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w78n9"]
Oct 31 07:29:07.349039 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.349009    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 07:29:07.349039 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.349026    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 07:29:07.349209 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.349045    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dvd7d\""
Oct 31 07:29:07.360431 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.360412    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfsf9\" (UniqueName: \"kubernetes.io/projected/b638fc1f-3579-467c-ac54-e4a4bc891f9e-kube-api-access-vfsf9\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:07.360470 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.360446    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b638fc1f-3579-467c-ac54-e4a4bc891f9e-tmp-dir\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:07.360566 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.360550    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05163761-47bd-4b1b-93fc-1ef20691a84f-cert\") pod \"ingress-canary-vktw2\" (UID: \"05163761-47bd-4b1b-93fc-1ef20691a84f\") " pod="openshift-ingress-canary/ingress-canary-vktw2"
Oct 31 07:29:07.360595 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.360574    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsbrt\" (UniqueName: \"kubernetes.io/projected/05163761-47bd-4b1b-93fc-1ef20691a84f-kube-api-access-vsbrt\") pod \"ingress-canary-vktw2\" (UID: \"05163761-47bd-4b1b-93fc-1ef20691a84f\") " pod="openshift-ingress-canary/ingress-canary-vktw2"
Oct 31 07:29:07.360595 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.360592    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:07.360659 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.360612    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b638fc1f-3579-467c-ac54-e4a4bc891f9e-config-volume\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:07.375417 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.375392    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:29:07.375513 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.375465    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:07.375513 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.375488    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:29:07.392561 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.392539    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 07:29:07.392561 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.392540    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 07:29:07.392749 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.392607    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 07:29:07.392749 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.392639    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 07:29:07.392918 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.392899    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-tdhmc\""
Oct 31 07:29:07.393125 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.392921    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-lzwlj\""
Oct 31 07:29:07.393693 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.393674    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w4rqx\""
Oct 31 07:29:07.461472 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.461412    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfsf9\" (UniqueName: \"kubernetes.io/projected/b638fc1f-3579-467c-ac54-e4a4bc891f9e-kube-api-access-vfsf9\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:07.461472 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.461452    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b638fc1f-3579-467c-ac54-e4a4bc891f9e-tmp-dir\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:07.461628 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.461501    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05163761-47bd-4b1b-93fc-1ef20691a84f-cert\") pod \"ingress-canary-vktw2\" (UID: \"05163761-47bd-4b1b-93fc-1ef20691a84f\") " pod="openshift-ingress-canary/ingress-canary-vktw2"
Oct 31 07:29:07.461628 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.461516    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsbrt\" (UniqueName: \"kubernetes.io/projected/05163761-47bd-4b1b-93fc-1ef20691a84f-kube-api-access-vsbrt\") pod \"ingress-canary-vktw2\" (UID: \"05163761-47bd-4b1b-93fc-1ef20691a84f\") " pod="openshift-ingress-canary/ingress-canary-vktw2"
Oct 31 07:29:07.461628 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.461533    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:07.461628 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.461557    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b638fc1f-3579-467c-ac54-e4a4bc891f9e-config-volume\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:07.461829 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:07.461677    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:07.461829 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:07.461762    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls podName:b638fc1f-3579-467c-ac54-e4a4bc891f9e nodeName:}" failed. No retries permitted until 2026-10-31 07:29:07.961739892 +0000 UTC m=+30.171074610 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls") pod "dns-default-w78n9" (UID: "b638fc1f-3579-467c-ac54-e4a4bc891f9e") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:07.461934 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.461848    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b638fc1f-3579-467c-ac54-e4a4bc891f9e-tmp-dir\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:07.462262 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.462222    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b638fc1f-3579-467c-ac54-e4a4bc891f9e-config-volume\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:07.465459 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.465342    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05163761-47bd-4b1b-93fc-1ef20691a84f-cert\") pod \"ingress-canary-vktw2\" (UID: \"05163761-47bd-4b1b-93fc-1ef20691a84f\") " pod="openshift-ingress-canary/ingress-canary-vktw2"
Oct 31 07:29:07.487136 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.487109    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfsf9\" (UniqueName: \"kubernetes.io/projected/b638fc1f-3579-467c-ac54-e4a4bc891f9e-kube-api-access-vfsf9\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:07.487456 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.487441    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsbrt\" (UniqueName: \"kubernetes.io/projected/05163761-47bd-4b1b-93fc-1ef20691a84f-kube-api-access-vsbrt\") pod \"ingress-canary-vktw2\" (UID: \"05163761-47bd-4b1b-93fc-1ef20691a84f\") " pod="openshift-ingress-canary/ingress-canary-vktw2"
Oct 31 07:29:07.542365 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.542344    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vktw2"
Oct 31 07:29:07.848063 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.846219    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vktw2"]
Oct 31 07:29:07.848167 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:29:07.848111    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05163761_47bd_4b1b_93fc_1ef20691a84f.slice/crio-5be6a30e6055de07f050a907898633d0f0bfe587d1686f45171dae6330c8476d WatchSource:0}: Error finding container 5be6a30e6055de07f050a907898633d0f0bfe587d1686f45171dae6330c8476d: Status 404 returned error can't find the container with id 5be6a30e6055de07f050a907898633d0f0bfe587d1686f45171dae6330c8476d
Oct 31 07:29:07.964608 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:07.964578    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:07.964782 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:07.964746    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:07.964852 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:07.964812    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls podName:b638fc1f-3579-467c-ac54-e4a4bc891f9e nodeName:}" failed. No retries permitted until 2026-10-31 07:29:08.964794166 +0000 UTC m=+31.174128890 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls") pod "dns-default-w78n9" (UID: "b638fc1f-3579-467c-ac54-e4a4bc891f9e") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:08.654834 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:08.654795    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vktw2" event={"ID":"05163761-47bd-4b1b-93fc-1ef20691a84f","Type":"ContainerStarted","Data":"5be6a30e6055de07f050a907898633d0f0bfe587d1686f45171dae6330c8476d"}
Oct 31 07:29:08.972450 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:08.972355    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:08.972608 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:08.972532    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:08.972608 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:08.972596    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls podName:b638fc1f-3579-467c-ac54-e4a4bc891f9e nodeName:}" failed. No retries permitted until 2026-10-31 07:29:10.972581027 +0000 UTC m=+33.181915744 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls") pod "dns-default-w78n9" (UID: "b638fc1f-3579-467c-ac54-e4a4bc891f9e") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:10.989458 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:10.989243    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:10.989895 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:10.989391    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:10.989895 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:10.989520    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls podName:b638fc1f-3579-467c-ac54-e4a4bc891f9e nodeName:}" failed. No retries permitted until 2026-10-31 07:29:14.98950685 +0000 UTC m=+37.198841554 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls") pod "dns-default-w78n9" (UID: "b638fc1f-3579-467c-ac54-e4a4bc891f9e") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:11.662350 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.662280    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vktw2" event={"ID":"05163761-47bd-4b1b-93fc-1ef20691a84f","Type":"ContainerStarted","Data":"f3f11cecebc9bf250f077e528c2554bf851b8a244d9ef67ecb3cb1e6ceba2ae8"}
Oct 31 07:29:11.707932 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.707885    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vktw2" podStartSLOduration=1.8844602099999999 podStartE2EDuration="4.707870527s" totalImagesPullingTime="2.823410317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:11.707271857 +0000 UTC m=+33.916606589" watchObservedRunningTime="2026-10-31 07:29:11.707870527 +0000 UTC m=+33.917205288"
Oct 31 07:29:11.845587 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.845556    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wqkwf"]
Oct 31 07:29:11.876465 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.876430    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wqkwf"]
Oct 31 07:29:11.876602 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.876546    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:11.893875 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.893854    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 07:29:11.893995 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.893887    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 07:29:11.893995 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.893948    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 07:29:11.894480 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.894466    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 07:29:11.894725 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.894714    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-48xxx\""
Oct 31 07:29:11.935047 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.934989    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-k76m7-4srkl"]
Oct 31 07:29:11.956816 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.956794    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:11.959138 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.959119    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-k76m7-4srkl"]
Oct 31 07:29:11.993896 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.993879    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-dl9sg\""
Oct 31 07:29:11.994189 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.993965    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 31 07:29:11.997708 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.997686    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f393f8a-63f4-41d7-aedd-d3df202eff35-data-volume\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:11.997786 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.997723    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:11.997786 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.997742    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:29:11.997876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.997802    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:11.997876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.997818    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsjgx\" (UniqueName: \"kubernetes.io/projected/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-api-access-zsjgx\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:11.997876 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.997857    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:11.998013 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:11.997884    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 07:29:11.998013 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:11.997931    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs podName:d95920d0-5a98-46bc-9b64-f820f02d5b1d nodeName:}" failed. No retries permitted until 2026-10-31 07:29:43.997918053 +0000 UTC m=+66.207252757 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs") pod "network-metrics-daemon-2skf7" (UID: "d95920d0-5a98-46bc-9b64-f820f02d5b1d") : secret "metrics-daemon-secret" not found
Oct 31 07:29:11.998013 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:11.997974    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f393f8a-63f4-41d7-aedd-d3df202eff35-crio-socket\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:12.001386 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.001367    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:12.023715 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.023685    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 31 07:29:12.098372 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.098345    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2978v\" (UniqueName: \"kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v\") pod \"network-check-target-pjddt\" (UID: \"72245812-04f1-47b8-a22f-67a61f175b63\") " pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:29:12.098496 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.098383    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcn6k\" (UniqueName: \"kubernetes.io/projected/deac7042-7310-4a66-98b3-88fbebb4f2f9-kube-api-access-pcn6k\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:12.098496 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.098403    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-trusted-ca-bundle\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:12.098496 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.098424    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/deac7042-7310-4a66-98b3-88fbebb4f2f9-archives-path\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:12.098496 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.098444    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f393f8a-63f4-41d7-aedd-d3df202eff35-crio-socket\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:12.098673 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.098546    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f393f8a-63f4-41d7-aedd-d3df202eff35-data-volume\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:12.098673 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.098585    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:12.098673 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.098605    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsjgx\" (UniqueName: \"kubernetes.io/projected/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-api-access-zsjgx\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:12.098673 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.098612    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f393f8a-63f4-41d7-aedd-d3df202eff35-crio-socket\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:12.098673 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.098640    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-service-ca-bundle\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:12.098673 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.098659    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:12.098943 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:12.098679    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 07:29:12.098943 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:12.098735    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm podName:8f393f8a-63f4-41d7-aedd-d3df202eff35 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:12.59872088 +0000 UTC m=+34.808055614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wqkwf" (UID: "8f393f8a-63f4-41d7-aedd-d3df202eff35") : configmap "kube-rbac-proxy" not found
Oct 31 07:29:12.098943 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:12.098750    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:12.098943 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:12.098795    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls podName:8f393f8a-63f4-41d7-aedd-d3df202eff35 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:12.598780494 +0000 UTC m=+34.808115198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqkwf" (UID: "8f393f8a-63f4-41d7-aedd-d3df202eff35") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:12.098943 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.098856    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f393f8a-63f4-41d7-aedd-d3df202eff35-data-volume\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:12.102620 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.100893    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2978v\" (UniqueName: \"kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v\") pod \"network-check-target-pjddt\" (UID: \"72245812-04f1-47b8-a22f-67a61f175b63\") " pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:29:12.124722 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.124704    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsjgx\" (UniqueName: \"kubernetes.io/projected/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-api-access-zsjgx\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:12.188491 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.188423    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:12.199839 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.199814    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-service-ca-bundle\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:12.199929 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.199859    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcn6k\" (UniqueName: \"kubernetes.io/projected/deac7042-7310-4a66-98b3-88fbebb4f2f9-kube-api-access-pcn6k\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:12.199929 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.199876    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-trusted-ca-bundle\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:12.199929 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.199897    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/deac7042-7310-4a66-98b3-88fbebb4f2f9-archives-path\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:12.200026 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.199954    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:29:12.200338 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.200313    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/deac7042-7310-4a66-98b3-88fbebb4f2f9-archives-path\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:12.200449 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.200433    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-service-ca-bundle\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:12.200769 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.200751    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-trusted-ca-bundle\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:12.235197 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.235173    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcn6k\" (UniqueName: \"kubernetes.io/projected/deac7042-7310-4a66-98b3-88fbebb4f2f9-kube-api-access-pcn6k\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:12.260575 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.260456    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-s2zkk"]
Oct 31 07:29:12.274103 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.271383    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-s2zkk"
Oct 31 07:29:12.281147 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.279710    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:12.284882 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.283893    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-s2zkk"]
Oct 31 07:29:12.289248 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.288931    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-9bh97\""
Oct 31 07:29:12.289674 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.289603    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 31 07:29:12.289674 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.289638    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 31 07:29:12.346314 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.345936    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-zn7pt"]
Oct 31 07:29:12.364380 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.364353    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:12.370266 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.370245    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-zn7pt"]
Oct 31 07:29:12.388850 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.388829    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-gt84m\""
Oct 31 07:29:12.388952 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.388940    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 31 07:29:12.389182 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.389168    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 31 07:29:12.389265 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.389222    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 31 07:29:12.389356 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.389343    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 31 07:29:12.402064 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.402041    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcsmv\" (UniqueName: \"kubernetes.io/projected/9ae7f604-ca2d-4ddd-8aac-3f3975ef7463-kube-api-access-tcsmv\") pod \"migrator-66df5d496b-s2zkk\" (UID: \"9ae7f604-ca2d-4ddd-8aac-3f3975ef7463\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-s2zkk"
Oct 31 07:29:12.503377 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.503305    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e9840048-339a-4d1a-a37c-cdfad40f8883-signing-key\") pod \"service-ca-5fffc97bdf-zn7pt\" (UID: \"e9840048-339a-4d1a-a37c-cdfad40f8883\") " pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:12.503377 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.503348    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e9840048-339a-4d1a-a37c-cdfad40f8883-signing-cabundle\") pod \"service-ca-5fffc97bdf-zn7pt\" (UID: \"e9840048-339a-4d1a-a37c-cdfad40f8883\") " pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:12.503377 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.503376    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl7mm\" (UniqueName: \"kubernetes.io/projected/e9840048-339a-4d1a-a37c-cdfad40f8883-kube-api-access-sl7mm\") pod \"service-ca-5fffc97bdf-zn7pt\" (UID: \"e9840048-339a-4d1a-a37c-cdfad40f8883\") " pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:12.503608 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.503468    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcsmv\" (UniqueName: \"kubernetes.io/projected/9ae7f604-ca2d-4ddd-8aac-3f3975ef7463-kube-api-access-tcsmv\") pod \"migrator-66df5d496b-s2zkk\" (UID: \"9ae7f604-ca2d-4ddd-8aac-3f3975ef7463\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-s2zkk"
Oct 31 07:29:12.507364 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:29:12.506855    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11dc8b0e_286d_42e1_bfe0_d882e7dfe073.slice/crio-d79ad42faa964e86d26e4e258cbdc586d82e52d412b60be7136c10b74449a743 WatchSource:0}: Error finding container d79ad42faa964e86d26e4e258cbdc586d82e52d412b60be7136c10b74449a743: Status 404 returned error can't find the container with id d79ad42faa964e86d26e4e258cbdc586d82e52d412b60be7136c10b74449a743
Oct 31 07:29:12.510165 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.510147    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z9h8d"]
Oct 31 07:29:12.529568 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.529546    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcsmv\" (UniqueName: \"kubernetes.io/projected/9ae7f604-ca2d-4ddd-8aac-3f3975ef7463-kube-api-access-tcsmv\") pod \"migrator-66df5d496b-s2zkk\" (UID: \"9ae7f604-ca2d-4ddd-8aac-3f3975ef7463\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-s2zkk"
Oct 31 07:29:12.537781 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.537757    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pjddt"]
Oct 31 07:29:12.537986 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:29:12.537968    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72245812_04f1_47b8_a22f_67a61f175b63.slice/crio-2cce4a82ce2acb5cc08f94741880dd8593b5ea7b5337e8cec54651c480361395 WatchSource:0}: Error finding container 2cce4a82ce2acb5cc08f94741880dd8593b5ea7b5337e8cec54651c480361395: Status 404 returned error can't find the container with id 2cce4a82ce2acb5cc08f94741880dd8593b5ea7b5337e8cec54651c480361395
Oct 31 07:29:12.586128 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.586099    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-k76m7-4srkl"]
Oct 31 07:29:12.592355 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:29:12.592334    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeac7042_7310_4a66_98b3_88fbebb4f2f9.slice/crio-04a7d71a2f31142041e2647831fe00ffed120ad1b4103003e326b8698aee6fa0 WatchSource:0}: Error finding container 04a7d71a2f31142041e2647831fe00ffed120ad1b4103003e326b8698aee6fa0: Status 404 returned error can't find the container with id 04a7d71a2f31142041e2647831fe00ffed120ad1b4103003e326b8698aee6fa0
Oct 31 07:29:12.603683 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.603663    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e9840048-339a-4d1a-a37c-cdfad40f8883-signing-cabundle\") pod \"service-ca-5fffc97bdf-zn7pt\" (UID: \"e9840048-339a-4d1a-a37c-cdfad40f8883\") " pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:12.603765 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.603697    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl7mm\" (UniqueName: \"kubernetes.io/projected/e9840048-339a-4d1a-a37c-cdfad40f8883-kube-api-access-sl7mm\") pod \"service-ca-5fffc97bdf-zn7pt\" (UID: \"e9840048-339a-4d1a-a37c-cdfad40f8883\") " pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:12.603765 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.603727    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:12.603839 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:12.603783    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 07:29:12.603839 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:12.603826    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm podName:8f393f8a-63f4-41d7-aedd-d3df202eff35 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:13.603815131 +0000 UTC m=+35.813149840 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wqkwf" (UID: "8f393f8a-63f4-41d7-aedd-d3df202eff35") : configmap "kube-rbac-proxy" not found
Oct 31 07:29:12.603918 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.603879    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:12.603964 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.603918    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e9840048-339a-4d1a-a37c-cdfad40f8883-signing-key\") pod \"service-ca-5fffc97bdf-zn7pt\" (UID: \"e9840048-339a-4d1a-a37c-cdfad40f8883\") " pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:12.604056 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.603986    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-s2zkk"
Oct 31 07:29:12.604056 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:12.604031    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:12.604171 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:12.604158    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls podName:8f393f8a-63f4-41d7-aedd-d3df202eff35 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:13.604139402 +0000 UTC m=+35.813474110 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqkwf" (UID: "8f393f8a-63f4-41d7-aedd-d3df202eff35") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:12.604396 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.604374    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e9840048-339a-4d1a-a37c-cdfad40f8883-signing-cabundle\") pod \"service-ca-5fffc97bdf-zn7pt\" (UID: \"e9840048-339a-4d1a-a37c-cdfad40f8883\") " pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:12.606193 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.606177    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e9840048-339a-4d1a-a37c-cdfad40f8883-signing-key\") pod \"service-ca-5fffc97bdf-zn7pt\" (UID: \"e9840048-339a-4d1a-a37c-cdfad40f8883\") " pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:12.628797 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.628774    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl7mm\" (UniqueName: \"kubernetes.io/projected/e9840048-339a-4d1a-a37c-cdfad40f8883-kube-api-access-sl7mm\") pod \"service-ca-5fffc97bdf-zn7pt\" (UID: \"e9840048-339a-4d1a-a37c-cdfad40f8883\") " pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:12.665735 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.665703    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pjddt" event={"ID":"72245812-04f1-47b8-a22f-67a61f175b63","Type":"ContainerStarted","Data":"2cce4a82ce2acb5cc08f94741880dd8593b5ea7b5337e8cec54651c480361395"}
Oct 31 07:29:12.666877 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.666848    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z9h8d" event={"ID":"11dc8b0e-286d-42e1-bfe0-d882e7dfe073","Type":"ContainerStarted","Data":"d79ad42faa964e86d26e4e258cbdc586d82e52d412b60be7136c10b74449a743"}
Oct 31 07:29:12.667918 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.667895    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-k76m7-4srkl" event={"ID":"deac7042-7310-4a66-98b3-88fbebb4f2f9","Type":"ContainerStarted","Data":"04a7d71a2f31142041e2647831fe00ffed120ad1b4103003e326b8698aee6fa0"}
Oct 31 07:29:12.689499 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.689479    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:12.951030 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.951006    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-s2zkk"]
Oct 31 07:29:12.954207 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:29:12.954174    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ae7f604_ca2d_4ddd_8aac_3f3975ef7463.slice/crio-05020db1dc97a26ed98891970b362b7028cae202d49b24ab7904f8897703477c WatchSource:0}: Error finding container 05020db1dc97a26ed98891970b362b7028cae202d49b24ab7904f8897703477c: Status 404 returned error can't find the container with id 05020db1dc97a26ed98891970b362b7028cae202d49b24ab7904f8897703477c
Oct 31 07:29:12.997379 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:12.997349    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-zn7pt"]
Oct 31 07:29:12.997959 ip-10-0-138-234 kubenswrapper[2572]: W1031 07:29:12.997938    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9840048_339a_4d1a_a37c_cdfad40f8883.slice/crio-1d4331e6aaff22c7074c470a5976c2336213eae4059b468255e4282c2bc9d3de WatchSource:0}: Error finding container 1d4331e6aaff22c7074c470a5976c2336213eae4059b468255e4282c2bc9d3de: Status 404 returned error can't find the container with id 1d4331e6aaff22c7074c470a5976c2336213eae4059b468255e4282c2bc9d3de
Oct 31 07:29:13.613672 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:13.612670    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:13.613672 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:13.612739    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:13.613672 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:13.612884    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:13.613672 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:13.612969    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls podName:8f393f8a-63f4-41d7-aedd-d3df202eff35 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:15.612950997 +0000 UTC m=+37.822285715 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqkwf" (UID: "8f393f8a-63f4-41d7-aedd-d3df202eff35") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:13.613672 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:13.613364    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 07:29:13.613672 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:13.613413    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm podName:8f393f8a-63f4-41d7-aedd-d3df202eff35 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:15.613398577 +0000 UTC m=+37.822733296 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wqkwf" (UID: "8f393f8a-63f4-41d7-aedd-d3df202eff35") : configmap "kube-rbac-proxy" not found
Oct 31 07:29:13.674655 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:13.674561    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt" event={"ID":"e9840048-339a-4d1a-a37c-cdfad40f8883","Type":"ContainerStarted","Data":"1d4331e6aaff22c7074c470a5976c2336213eae4059b468255e4282c2bc9d3de"}
Oct 31 07:29:13.676675 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:13.676608    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-s2zkk" event={"ID":"9ae7f604-ca2d-4ddd-8aac-3f3975ef7463","Type":"ContainerStarted","Data":"05020db1dc97a26ed98891970b362b7028cae202d49b24ab7904f8897703477c"}
Oct 31 07:29:15.025625 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:15.025405    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:15.026000 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:15.025658    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:15.026000 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:15.025733    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls podName:b638fc1f-3579-467c-ac54-e4a4bc891f9e nodeName:}" failed. No retries permitted until 2026-10-31 07:29:23.025710934 +0000 UTC m=+45.235045654 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls") pod "dns-default-w78n9" (UID: "b638fc1f-3579-467c-ac54-e4a4bc891f9e") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:15.631487 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:15.631455    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:15.631685 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:15.631527    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:15.631685 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:15.631594    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 07:29:15.631685 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:15.631667    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:15.631865 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:15.631696    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm podName:8f393f8a-63f4-41d7-aedd-d3df202eff35 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:19.63167515 +0000 UTC m=+41.841009874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wqkwf" (UID: "8f393f8a-63f4-41d7-aedd-d3df202eff35") : configmap "kube-rbac-proxy" not found
Oct 31 07:29:15.631865 ip-10-0-138-234 kubenswrapper[2572]: E1031 07:29:15.631734    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls podName:8f393f8a-63f4-41d7-aedd-d3df202eff35 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:19.631717097 +0000 UTC m=+41.841051824 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqkwf" (UID: "8f393f8a-63f4-41d7-aedd-d3df202eff35") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:15.676949 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:15.676858    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ll6x_e07914e4-a5e8-42b8-889a-5fab59642856/konnectivity-agent/0.log" limit=1024
Oct 31 07:29:15.686076 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:15.686039    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-s2zkk" event={"ID":"9ae7f604-ca2d-4ddd-8aac-3f3975ef7463","Type":"ContainerStarted","Data":"ae832dc127983ff2f0dc27c66cf63b2a28b190b9177d4ad954a0b25a7e1baf75"}
Oct 31 07:29:15.731605 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:15.731545    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-s2zkk" podStartSLOduration=1.687287762 podStartE2EDuration="3.731525579s" totalImagesPullingTime="2.044237817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:15.730448061 +0000 UTC m=+37.939782798" watchObservedRunningTime="2026-10-31 07:29:15.731525579 +0000 UTC m=+37.940860308"
Oct 31 07:29:16.491460 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:16.491387    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ll6x_e07914e4-a5e8-42b8-889a-5fab59642856/konnectivity-agent/0.log" limit=1024
Oct 31 07:29:16.691343 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:16.690871    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt" event={"ID":"e9840048-339a-4d1a-a37c-cdfad40f8883","Type":"ContainerStarted","Data":"a70006c72f3db74f8cdcad9bdcc555f4ee67f74efd83f59844e68a43153aeb08"}
Oct 31 07:29:16.695534 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:16.693644    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-k76m7-4srkl" event={"ID":"deac7042-7310-4a66-98b3-88fbebb4f2f9","Type":"ContainerStarted","Data":"6e478c628c8cb2bbcac22ce4eeceafc962c8f873266a3b0aad09838d25225927"}
Oct 31 07:29:16.738509 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:16.738410    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt" podStartSLOduration=1.256759526 podStartE2EDuration="4.738390285s" totalImagesPullingTime="3.481630759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:16.737345626 +0000 UTC m=+38.946680352" watchObservedRunningTime="2026-10-31 07:29:16.738390285 +0000 UTC m=+38.947725013"
Oct 31 07:29:16.784913 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:16.784855    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-k76m7-4srkl" podStartSLOduration=2.267589321 podStartE2EDuration="5.784837294s" totalImagesPullingTime="3.517247973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:16.783911301 +0000 UTC m=+38.993246030" watchObservedRunningTime="2026-10-31 07:29:16.784837294 +0000 UTC m=+38.994172023"
Oct 31 07:29:17.698423 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:17.698141    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pjddt" event={"ID":"72245812-04f1-47b8-a22f-67a61f175b63","Type":"ContainerStarted","Data":"83b4bc5f7d5ca43c5b9d41e5235964d5026ad0da8039aca39f6c244719b4b5c1"}
Oct 31 07:29:17.699013 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:17.698593    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:29:17.744119 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:17.744060    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pjddt" podStartSLOduration=35.445893954 podStartE2EDuration="39.744046967s" totalImagesPullingTime="4.298153013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:17.74363701 +0000 UTC m=+39.952971730" watchObservedRunningTime="2026-10-31 07:29:17.744046967 +0000 UTC m=+39.953381693"
Oct 31 07:29:18.673700 ip-10-0-138-234 kubenswrapper[2572]: I1031 07:29:18.673614    2572 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 07:29:18.673776 ip-10-0-138-234 systemd[1]: Stopping Kubernetes Kubelet...
Oct 31 07:29:18.690914 ip-10-0-138-234 systemd[1]: kubelet.service: Deactivated successfully.
Oct 31 07:29:18.691081 ip-10-0-138-234 systemd[1]: Stopped Kubernetes Kubelet.
Oct 31 07:29:18.691259 ip-10-0-138-234 systemd[1]: kubelet.service: Consumed 2.361s CPU time.
Oct 31 07:29:18.706688 ip-10-0-138-234 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 07:29:18.793824 ip-10-0-138-234 kubenswrapper[4322]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 07:29:18.793824 ip-10-0-138-234 kubenswrapper[4322]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 07:29:18.793824 ip-10-0-138-234 kubenswrapper[4322]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 07:29:18.793824 ip-10-0-138-234 kubenswrapper[4322]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 07:29:18.795889 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795818    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:18.795889 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795888    4322 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:18.795889 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795892    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795895    4322 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795898    4322 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795901    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795903    4322 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795906    4322 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795909    4322 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795911    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795914    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795917    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795919    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795922    4322 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795925    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795927    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795930    4322 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795932    4322 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795935    4322 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795937    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795940    4322 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:18.795978 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795942    4322 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795945    4322 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795947    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795950    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795961    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795964    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795966    4322 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795969    4322 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795971    4322 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795974    4322 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795977    4322 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795979    4322 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795982    4322 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795985    4322 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795988    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795992    4322 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.795996    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796000    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796002    4322 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:18.796497 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796005    4322 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796008    4322 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796011    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796013    4322 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796016    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796018    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796021    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796024    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796026    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796029    4322 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796031    4322 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796034    4322 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796036    4322 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796039    4322 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796045    4322 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796048    4322 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796051    4322 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796053    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796056    4322 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796058    4322 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:18.796954 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796061    4322 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796064    4322 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796066    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796069    4322 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796072    4322 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796074    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796077    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796079    4322 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796082    4322 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796084    4322 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796087    4322 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796089    4322 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796091    4322 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796094    4322 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796096    4322 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796099    4322 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796102    4322 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796105    4322 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796108    4322 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:18.797432 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796111    4322 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796113    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796118    4322 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796122    4322 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796125    4322 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796128    4322 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796131    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796135    4322 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796138    4322 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796141    4322 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796143    4322 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796146    4322 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796148    4322 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796151    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796153    4322 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796156    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796158    4322 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796161    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796164    4322 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796166    4322 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:18.797872 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796169    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796171    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796174    4322 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796177    4322 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796179    4322 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796182    4322 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796184    4322 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796986    4322 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796993    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.796997    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797000    4322 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797003    4322 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797005    4322 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797008    4322 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797011    4322 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797013    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797016    4322 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797018    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797021    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:18.798369 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797026    4322 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797029    4322 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797032    4322 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797034    4322 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797037    4322 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797040    4322 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797042    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797045    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797048    4322 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797050    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797054    4322 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797057    4322 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797060    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797062    4322 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797065    4322 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797068    4322 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797070    4322 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797073    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797076    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797078    4322 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:18.798830 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797081    4322 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797083    4322 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797086    4322 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797089    4322 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797093    4322 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797097    4322 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797100    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797102    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797105    4322 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797107    4322 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797110    4322 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797112    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797116    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797119    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797122    4322 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797124    4322 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797127    4322 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797130    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797133    4322 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797135    4322 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:18.799325 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797138    4322 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797140    4322 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797144    4322 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797146    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797149    4322 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797151    4322 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797154    4322 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797157    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797159    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797162    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797164    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797167    4322 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797170    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797172    4322 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797175    4322 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797177    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797180    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797182    4322 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797185    4322 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:18.799801 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797187    4322 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797190    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797192    4322 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797197    4322 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797200    4322 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797204    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797207    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797210    4322 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797213    4322 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797216    4322 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797218    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797220    4322 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797236    4322 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797239    4322 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797242    4322 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797245    4322 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797248    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797251    4322 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797253    4322 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:18.800274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797256    4322 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797259    4322 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797261    4322 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797264    4322 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797266    4322 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797269    4322 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797271    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797274    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797276    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797279    4322 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797282    4322 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797284    4322 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797287    4322 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797289    4322 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797291    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797294    4322 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797754    4322 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797759    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797763    4322 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797766    4322 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:18.800739 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797769    4322 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797772    4322 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797775    4322 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797777    4322 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797780    4322 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797782    4322 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797785    4322 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797787    4322 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797791    4322 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797793    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797796    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797799    4322 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797801    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797804    4322 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797806    4322 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797809    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797811    4322 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797814    4322 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797817    4322 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:18.801222 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797821    4322 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797824    4322 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797826    4322 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797829    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797833    4322 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797836    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797839    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797842    4322 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797845    4322 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797847    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797850    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797854    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797857    4322 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797860    4322 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797862    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797865    4322 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797868    4322 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797870    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797873    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:18.801783 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797875    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797878    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797881    4322 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797883    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797886    4322 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797889    4322 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797891    4322 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797893    4322 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797896    4322 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797899    4322 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797901    4322 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797903    4322 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797906    4322 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797909    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797911    4322 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797914    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797917    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797919    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797922    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797924    4322 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:18.802237 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797927    4322 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797930    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797932    4322 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797935    4322 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797938    4322 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797940    4322 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797943    4322 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797946    4322 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797949    4322 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797951    4322 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797954    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797956    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797959    4322 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797961    4322 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797964    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797966    4322 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797969    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797971    4322 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797974    4322 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797976    4322 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:18.802736 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797979    4322 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797981    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797984    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797986    4322 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797989    4322 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797991    4322 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797994    4322 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797996    4322 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.797999    4322 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.798001    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.798003    4322 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.798006    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.798009    4322 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.798012    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.798014    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.798016    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.798019    4322 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.798022    4322 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.798024    4322 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.798028    4322 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:18.803465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.798031    4322 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.798034    4322 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.798036    4322 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.798039    4322 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798144    4322 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798154    4322 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798164    4322 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798171    4322 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798177    4322 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798182    4322 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798188    4322 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798195    4322 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798200    4322 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798205    4322 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798211    4322 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798216    4322 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798235    4322 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798241    4322 flags.go:64] FLAG: --cgroup-root=""
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798245    4322 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798250    4322 flags.go:64] FLAG: --client-ca-file=""
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798255    4322 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798259    4322 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798266    4322 flags.go:64] FLAG: --cluster-domain=""
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798270    4322 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 07:29:18.804136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798275    4322 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798280    4322 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798286    4322 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798292    4322 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798297    4322 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798302    4322 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798307    4322 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798312    4322 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798317    4322 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798323    4322 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798329    4322 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798333    4322 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798340    4322 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798345    4322 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798349    4322 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798354    4322 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798359    4322 flags.go:64] FLAG: --enable-server="true"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798364    4322 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798371    4322 flags.go:64] FLAG: --event-burst="100"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798376    4322 flags.go:64] FLAG: --event-qps="50"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798381    4322 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798385    4322 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798390    4322 flags.go:64] FLAG: --eviction-hard=""
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798396    4322 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 07:29:18.804839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798403    4322 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798407    4322 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798412    4322 flags.go:64] FLAG: --eviction-soft=""
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798417    4322 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798422    4322 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798427    4322 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798432    4322 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798437    4322 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798441    4322 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798446    4322 flags.go:64] FLAG: --feature-gates=""
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798452    4322 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798457    4322 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798462    4322 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798467    4322 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798472    4322 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798477    4322 flags.go:64] FLAG: --help="false"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798481    4322 flags.go:64] FLAG: --hostname-override="ip-10-0-138-234.ec2.internal"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798486    4322 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798492    4322 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798497    4322 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798502    4322 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798507    4322 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798512    4322 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 07:29:18.805482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798516    4322 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798521    4322 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798526    4322 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798530    4322 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798535    4322 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798540    4322 flags.go:64] FLAG: --kube-reserved=""
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798544    4322 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798549    4322 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798554    4322 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798560    4322 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798564    4322 flags.go:64] FLAG: --lock-file=""
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798569    4322 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798574    4322 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798578    4322 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798587    4322 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798592    4322 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798596    4322 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798601    4322 flags.go:64] FLAG: --logging-format="text"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798607    4322 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798612    4322 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798617    4322 flags.go:64] FLAG: --manifest-url=""
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798622    4322 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798628    4322 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798633    4322 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798639    4322 flags.go:64] FLAG: --max-pods="110"
Oct 31 07:29:18.806032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798644    4322 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798649    4322 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798654    4322 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798659    4322 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798664    4322 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798669    4322 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798674    4322 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798684    4322 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798688    4322 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798693    4322 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798698    4322 flags.go:64] FLAG: --pod-cidr=""
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798703    4322 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798708    4322 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798713    4322 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798717    4322 flags.go:64] FLAG: --port="10250"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798722    4322 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798728    4322 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f35e7a779a2e043e"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798734    4322 flags.go:64] FLAG: --qos-reserved=""
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798738    4322 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798743    4322 flags.go:64] FLAG: --register-node="true"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798748    4322 flags.go:64] FLAG: --register-with-taints=""
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798753    4322 flags.go:64] FLAG: --registry-burst="10"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798758    4322 flags.go:64] FLAG: --registry-qps="5"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798763    4322 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798767    4322 flags.go:64] FLAG: --reserved-memory=""
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798775    4322 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 07:29:18.806650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798780    4322 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798784    4322 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798789    4322 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798794    4322 flags.go:64] FLAG: --runonce="false"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798798    4322 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798803    4322 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798808    4322 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798812    4322 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798818    4322 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798822    4322 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798828    4322 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798833    4322 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798838    4322 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798843    4322 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798847    4322 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798852    4322 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798857    4322 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798861    4322 flags.go:64] FLAG: --system-cgroups=""
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798866    4322 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798874    4322 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798878    4322 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798883    4322 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798890    4322 flags.go:64] FLAG: --tls-min-version=""
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798895    4322 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798900    4322 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 07:29:18.807421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798904    4322 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798909    4322 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798913    4322 flags.go:64] FLAG: --v="2"
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798920    4322 flags.go:64] FLAG: --version="false"
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798926    4322 flags.go:64] FLAG: --vmodule=""
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798932    4322 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.798939    4322 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799111    4322 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799116    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799120    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799122    4322 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799125    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799128    4322 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799131    4322 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799133    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799136    4322 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799139    4322 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799141    4322 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799144    4322 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799147    4322 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799150    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:18.808021 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799153    4322 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799155    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799158    4322 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799161    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799163    4322 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799167    4322 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799171    4322 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799174    4322 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799179    4322 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799182    4322 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799184    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799187    4322 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799190    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799192    4322 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799195    4322 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799197    4322 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799200    4322 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799204    4322 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799206    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:18.808578 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799209    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799211    4322 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799214    4322 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799217    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799219    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799222    4322 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799236    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799238    4322 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799241    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799244    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799246    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799249    4322 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799252    4322 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799254    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799257    4322 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799260    4322 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799262    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799265    4322 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799267    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:18.809023 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799270    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799273    4322 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799277    4322 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799279    4322 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799282    4322 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799284    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799287    4322 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799289    4322 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799292    4322 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799294    4322 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799297    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799302    4322 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799305    4322 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799307    4322 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799310    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799312    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799315    4322 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799318    4322 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799321    4322 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799323    4322 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:18.809484 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799326    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799329    4322 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799331    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799334    4322 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799337    4322 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799341    4322 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799345    4322 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799348    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799351    4322 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799353    4322 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799356    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799358    4322 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799361    4322 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799363    4322 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799367    4322 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799370    4322 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799372    4322 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799375    4322 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799377    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:18.809999 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799380    4322 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799382    4322 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799385    4322 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799388    4322 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799392    4322 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799395    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799397    4322 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799400    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799403    4322 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799405    4322 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799408    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799410    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799413    4322 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799415    4322 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799418    4322 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.799445    4322 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 07:29:18.810558 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799479    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799483    4322 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799486    4322 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799490    4322 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799492    4322 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799495    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799498    4322 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799501    4322 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799503    4322 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799505    4322 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799508    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799512    4322 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799515    4322 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799517    4322 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799520    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799523    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799526    4322 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799529    4322 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799531    4322 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799534    4322 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:18.811029 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799537    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799540    4322 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799543    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799545    4322 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799548    4322 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799552    4322 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799555    4322 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799558    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799561    4322 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799564    4322 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799567    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799569    4322 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799572    4322 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799574    4322 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799577    4322 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799579    4322 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799582    4322 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799585    4322 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799588    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:18.811570 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799590    4322 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799593    4322 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799595    4322 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799598    4322 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799602    4322 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799604    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799607    4322 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799609    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799612    4322 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799615    4322 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799618    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799620    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799623    4322 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799625    4322 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799629    4322 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799632    4322 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799635    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799637    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799640    4322 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799642    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:18.812015 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799645    4322 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799647    4322 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799650    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799653    4322 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799656    4322 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799659    4322 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799661    4322 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799664    4322 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799666    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799669    4322 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799672    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799675    4322 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799677    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799680    4322 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799684    4322 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799687    4322 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799691    4322 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799694    4322 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799697    4322 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:18.812662 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799699    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799702    4322 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799704    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799707    4322 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799709    4322 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799712    4322 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799714    4322 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799718    4322 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799721    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799724    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799726    4322 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799730    4322 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799732    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799735    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799738    4322 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799740    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799743    4322 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799745    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799748    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:18.813465 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799750    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799753    4322 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799755    4322 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799758    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799760    4322 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799763    4322 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799766    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799768    4322 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.799771    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.805350    4322 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.805365    4322 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.805384    4322 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.805389    4322 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805447    4322 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805452    4322 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805455    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805459    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805462    4322 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805465    4322 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:18.814019 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805468    4322 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805472    4322 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805474    4322 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805477    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805480    4322 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805482    4322 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805485    4322 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805488    4322 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805490    4322 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805493    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805496    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805499    4322 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805502    4322 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805505    4322 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805507    4322 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805510    4322 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805512    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805515    4322 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805517    4322 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:18.814496 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805520    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805523    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805525    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805528    4322 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805530    4322 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805533    4322 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805536    4322 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805539    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805541    4322 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805544    4322 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805546    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805549    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805551    4322 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805554    4322 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805556    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805559    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805561    4322 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805564    4322 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805567    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:18.814947 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805569    4322 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805572    4322 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805576    4322 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805578    4322 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805581    4322 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805583    4322 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805586    4322 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805589    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805591    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805594    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805596    4322 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805599    4322 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805601    4322 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805604    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805606    4322 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805608    4322 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805611    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805614    4322 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805616    4322 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805619    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:18.815409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805621    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805623    4322 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805626    4322 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805630    4322 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805634    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805637    4322 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805640    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805643    4322 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805645    4322 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805648    4322 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805651    4322 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805653    4322 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805656    4322 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805658    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805661    4322 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805664    4322 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805666    4322 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805669    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805671    4322 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:18.815936 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805674    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805676    4322 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805679    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805681    4322 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805684    4322 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805686    4322 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805688    4322 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805691    4322 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805693    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805696    4322 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805699    4322 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805703    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805706    4322 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805709    4322 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805712    4322 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805715    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805717    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805720    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805722    4322 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805724    4322 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:18.816530 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805727    4322 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805730    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805732    4322 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.805758    4322 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805902    4322 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805908    4322 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805911    4322 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805914    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805917    4322 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805920    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805923    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805926    4322 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805929    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805932    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805934    4322 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805937    4322 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:18.817149 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805939    4322 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805942    4322 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805945    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805947    4322 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805949    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805952    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805955    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805957    4322 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805960    4322 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805962    4322 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805965    4322 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805967    4322 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805969    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805972    4322 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805974    4322 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805977    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805979    4322 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805982    4322 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805984    4322 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805987    4322 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:18.817664 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805989    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805992    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805994    4322 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.805997    4322 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806000    4322 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806002    4322 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806005    4322 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806008    4322 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806010    4322 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806013    4322 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806015    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806018    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806020    4322 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806023    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806039    4322 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806043    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806046    4322 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806049    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806051    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806054    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:18.818150 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806056    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806059    4322 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806062    4322 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806064    4322 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806066    4322 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806069    4322 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806072    4322 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806076    4322 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806078    4322 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806081    4322 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806084    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806088    4322 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806092    4322 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806095    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806098    4322 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806101    4322 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806104    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806107    4322 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806110    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:18.818743 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806115    4322 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806118    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806120    4322 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806124    4322 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806128    4322 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806131    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806133    4322 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806136    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806138    4322 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806143    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806146    4322 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806148    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806151    4322 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806153    4322 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806156    4322 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806158    4322 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806161    4322 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806163    4322 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806166    4322 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:18.819306 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806168    4322 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806172    4322 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806175    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806177    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806180    4322 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806182    4322 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806185    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806188    4322 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806190    4322 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806192    4322 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806195    4322 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806198    4322 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806200    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806203    4322 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806206    4322 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806209    4322 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:18.820079 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.806245    4322 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806288    4322 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806292    4322 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806295    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806298    4322 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806301    4322 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806305    4322 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806308    4322 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806311    4322 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806313    4322 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806316    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806319    4322 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806321    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806324    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806327    4322 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806330    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806332    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806335    4322 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806339    4322 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806341    4322 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:18.820678 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806344    4322 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806354    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806358    4322 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806361    4322 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806364    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806367    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806370    4322 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806372    4322 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806374    4322 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806377    4322 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806380    4322 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806383    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806385    4322 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806388    4322 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806390    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806393    4322 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806395    4322 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806398    4322 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806402    4322 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:18.821170 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806404    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806407    4322 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806409    4322 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806412    4322 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806414    4322 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806417    4322 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806419    4322 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806422    4322 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806424    4322 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806427    4322 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806429    4322 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806433    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806436    4322 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806439    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806441    4322 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806444    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806447    4322 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806450    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806453    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806456    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:18.821700 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806458    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806460    4322 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806463    4322 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806466    4322 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806468    4322 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806471    4322 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806474    4322 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806477    4322 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806479    4322 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806482    4322 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806484    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806490    4322 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806494    4322 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806498    4322 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806501    4322 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806504    4322 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806507    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806510    4322 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806513    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:18.822183 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806515    4322 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806518    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806520    4322 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806524    4322 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806529    4322 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806534    4322 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806538    4322 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806542    4322 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806546    4322 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806550    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806554    4322 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806558    4322 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806562    4322 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806566    4322 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806569    4322 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806571    4322 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806574    4322 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806577    4322 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806579    4322 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:18.822649 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806583    4322 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806586    4322 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806589    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806592    4322 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806595    4322 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806599    4322 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806602    4322 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806604    4322 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806607    4322 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:18.806609    4322 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.806658    4322 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.807674    4322 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.807731    4322 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.807898    4322 server.go:1014] "Starting client certificate rotation"
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.808047    4322 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.808090    4322 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 07:23:38 +0000 UTC" deadline="2028-06-05 17:06:39.408107955 +0000 UTC"
Oct 31 07:29:18.823100 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.808127    4322 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14001h37m20.599985005s"
Oct 31 07:29:18.823502 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.808381    4322 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 07:29:18.823502 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.809380    4322 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 07:29:18.823502 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.810743    4322 log.go:25] "Validated CRI v1 runtime API"
Oct 31 07:29:18.823502 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.812284    4322 log.go:25] "Validated CRI v1 image API"
Oct 31 07:29:18.823502 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.813025    4322 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 07:29:18.823502 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.814867    4322 fs.go:135] Filesystem UUIDs: map[1dcdf532-963a-48eb-9e1b-cae16f327670:/dev/nvme0n1p4 55737d7b-3d13-4d35-a487-76872c3424c6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Oct 31 07:29:18.823661 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.814886    4322 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/04a7d71a2f31142041e2647831fe00ffed120ad1b4103003e326b8698aee6fa0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/04a7d71a2f31142041e2647831fe00ffed120ad1b4103003e326b8698aee6fa0/userdata/shm major:0 minor:220 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/05020db1dc97a26ed98891970b362b7028cae202d49b24ab7904f8897703477c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/05020db1dc97a26ed98891970b362b7028cae202d49b24ab7904f8897703477c/userdata/shm major:0 minor:231 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1306f68e8130e53948cb2eb356fd5d7930dac4bbc36095df328e7375bc5ffdea/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1306f68e8130e53948cb2eb356fd5d7930dac4bbc36095df328e7375bc5ffdea/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1d4331e6aaff22c7074c470a5976c2336213eae4059b468255e4282c2bc9d3de/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1d4331e6aaff22c7074c470a5976c2336213eae4059b468255e4282c2bc9d3de/userdata/shm major:0 minor:233 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2cce4a82ce2acb5cc08f94741880dd8593b5ea7b5337e8cec54651c480361395/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2cce4a82ce2acb5cc08f94741880dd8593b5ea7b5337e8cec54651c480361395/userdata/shm major:0 minor:218 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/35807caf7071ae80511bf71e7553e3f12001be5ddf3f0ae1c51c87e5fbacbb16/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/35807caf7071ae80511bf71e7553e3f12001be5ddf3f0ae1c51c87e5fbacbb16/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5be6a30e6055de07f050a907898633d0f0bfe587d1686f45171dae6330c8476d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5be6a30e6055de07f050a907898633d0f0bfe587d1686f45171dae6330c8476d/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6880eb0e8fe90c744e2d7d2097a3361b4478005f3e29885f235f64bdc4ca5303/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6880eb0e8fe90c744e2d7d2097a3361b4478005f3e29885f235f64bdc4ca5303/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6fff91a9f5dad61d1b0a19109961e99874b1dd5f8faa562d54b9cb23426ae956/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6fff91a9f5dad61d1b0a19109961e99874b1dd5f8faa562d54b9cb23426ae956/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/828e47b40abf587bf9f22ade36b0923d9911ce9d6a1641a9af54fe8f7ae953f8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/828e47b40abf587bf9f22ade36b0923d9911ce9d6a1641a9af54fe8f7ae953f8/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/83315d191c3d017dcefd0d2fe3ee667ae8e2d9d62dacc42a89fedbb045e39319/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/83315d191c3d017dcefd0d2fe3ee667ae8e2d9d62dacc42a89fedbb045e39319/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8c73287f7d0aa2a7cbd56a488b7d4e9cbd5dfd0549c78c5610c4d4581d8e89e0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8c73287f7d0aa2a7cbd56a488b7d4e9cbd5dfd0549c78c5610c4d4581d8e89e0/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9ace0b66dfed7d90a6eafcab8506be3c744bf165e2104be4070aa7e2cf55744a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9ace0b66dfed7d90a6eafcab8506be3c744bf165e2104be4070aa7e2cf55744a/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bf9240bcbb27028905a2df9dc22ad44f2f83a0db479f52303bd11a643739d80a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bf9240bcbb27028905a2df9dc22ad44f2f83a0db479f52303bd11a643739d80a/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d79ad42faa964e86d26e4e258cbdc586d82e52d412b60be7136c10b74449a743/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d79ad42faa964e86d26e4e258cbdc586d82e52d412b60be7136c10b74449a743/userdata/shm major:0 minor:215 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/daacb703ff4a2581c23b6baa731c6c0428df334169ba8f0b6a522387e75f9e71/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/daacb703ff4a2581c23b6baa731c6c0428df334169ba8f0b6a522387e75f9e71/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/05163761-47bd-4b1b-93fc-1ef20691a84f/volumes/kubernetes.io~projected/kube-api-access-vsbrt:{mountpoint:/var/lib/kubelet/pods/05163761-47bd-4b1b-93fc-1ef20691a84f/volumes/kubernetes.io~projected/kube-api-access-vsbrt major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/05163761-47bd-4b1b-93fc-1ef20691a84f/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/05163761-47bd-4b1b-93fc-1ef20691a84f/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0f24a74d-99e3-4bce-a9e1-2eb550655093/volumes/kubernetes.io~projected/kube-api-access-75t9x:{mountpoint:/var/lib/kubelet/pods/0f24a74d-99e3-4bce-a9e1-2eb550655093/volumes/kubernetes.io~projected/kube-api-access-75t9x major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11dc8b0e-286d-42e1-bfe0-d882e7dfe073/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/11dc8b0e-286d-42e1-bfe0-d882e7dfe073/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a089cc1-9715-4809-9479-02fddca15479/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/1a089cc1-9715-4809-9479-02fddca15479/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a089cc1-9715-4809-9479-02fddca15479/volumes/kubernetes.io~projected/kube-api-access-xtwtf:{mountpoint:/var/lib/kubelet/pods/1a089cc1-9715-4809-9479-02fddca15479/volumes/kubernetes.io~projected/kube-api-access-xtwtf major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a089cc1-9715-4809-9479-02fddca15479/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/1a089cc1-9715-4809-9479-02fddca15479/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/374fae1c-44be-4729-802f-e190816bf249/volumes/kubernetes.io~projected/kube-api-access-8vt2p:{mountpoint:/var/lib/kubelet/pods/374fae1c-44be-4729-802f-e190816bf249/volumes/kubernetes.io~projected/kube-api-access-8vt2p major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62191e70-8059-4950-b636-b6c58d33b128/volumes/kubernetes.io~projected/kube-api-access-979pp:{mountpoint:/var/lib/kubelet/pods/62191e70-8059-4950-b636-b6c58d33b128/volumes/kubernetes.io~projected/kube-api-access-979pp major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/72245812-04f1-47b8-a22f-67a61f175b63/volumes/kubernetes.io~projected/kube-api-access-2978v:{mountpoint:/var/lib/kubelet/pods/72245812-04f1-47b8-a22f-67a61f175b63/volumes/kubernetes.io~projected/kube-api-access-2978v major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/881291b2-1e3f-491d-8dd8-b1500fc5ab24/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/881291b2-1e3f-491d-8dd8-b1500fc5ab24/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/881291b2-1e3f-491d-8dd8-b1500fc5ab24/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/881291b2-1e3f-491d-8dd8-b1500fc5ab24/volumes/kubernetes.io~empty-dir/tmp major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/881291b2-1e3f-491d-8dd8-b1500fc5ab24/volumes/kubernetes.io~projected/kube-api-access-kbq66:{mountpoint:/var/lib/kubelet/pods/881291b2-1e3f-491d-8dd8-b1500fc5ab24/volumes/kubernetes.io~projected/kube-api-access-kbq66 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8f393f8a-63f4-41d7-aedd-d3df202eff35/volumes/kubernetes.io~projected/kube-api-access-zsjgx:{mountpoint:/var/lib/kubelet/pods/8f393f8a-63f4-41d7-aedd-d3df202eff35/volumes/kubernetes.io~projected/kube-api-access-zsjgx major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ae7f604-ca2d-4ddd-8aac-3f3975ef7463/volumes/kubernetes.io~projected/kube-api-access-tcsmv:{mountpoint:/var/lib/kubelet/pods/9ae7f604-ca2d-4ddd-8aac-3f3975ef7463/volumes/kubernetes.io~projected/kube-api-access-tcsmv major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b1ca5663-7cde-4b55-b971-e2afaf82fb5f/volumes/kubernetes.io~projected/kube-api-access-699ww:{mountpoint:/var/lib/kubelet/pods/b1ca5663-7cde-4b55-b971-e2afaf82fb5f/volumes/kubernetes.io~projected/kube-api-access-699ww major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b638fc1f-3579-467c-ac54-e4a4bc891f9e/volumes/kubernetes.io~projected/kube-api-access-vfsf9:{mountpoint:/var/lib/kubelet/pods/b638fc1f-3579-467c-ac54-e4a4bc891f9e/volumes/kubernetes.io~projected/kube-api-access-vfsf9 major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf2415aa-becd-409e-bb51-05d94a380deb/volumes/kubernetes.io~projected/kube-api-access-5624d:{mountpoint:/var/lib/kubelet/pods/cf2415aa-becd-409e-bb51-05d94a380deb/volumes/kubernetes.io~projected/kube-api-access-5624d major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d95920d0-5a98-46bc-9b64-f820f02d5b1d/volumes/kubernetes.io~projected/kube-api-access-n4s6h:{mountpoint:/var/lib/kubelet/pods/d95920d0-5a98-46bc-9b64-f820f02d5b1d/volumes/kubernetes.io~projected/kube-api-access-n4s6h major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/deac7042-7310-4a66-98b3-88fbebb4f2f9/volumes/kubernetes.io~projected/kube-api-access-pcn6k:{mountpoint:/var/lib/kubelet/pods/deac7042-7310-4a66-98b3-88fbebb4f2f9/volumes/kubernetes.io~projected/kube-api-access-pcn6k major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e07914e4-a5e8-42b8-889a-5fab59642856/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/e07914e4-a5e8-42b8-889a-5fab59642856/volumes/kubernetes.io~secret/agent-certs major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e9840048-339a-4d1a-a37c-cdfad40f8883/volumes/kubernetes.io~projected/kube-api-access-sl7mm:{mountpoint:/var/lib/kubelet/pods/e9840048-339a-4d1a-a37c-cdfad40f8883/volumes/kubernetes.io~projected/kube-api-access-sl7mm major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e9840048-339a-4d1a-a37c-cdfad40f8883/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/e9840048-339a-4d1a-a37c-cdfad40f8883/volumes/kubernetes.io~secret/signing-key major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed269b7c-e48c-4732-acc4-c0250a95801f/volumes/kubernetes.io~projected/kube-api-access-4xzg5:{mountpoint:/var/lib/kubelet/pods/ed269b7c-e48c-4732-acc4-c0250a95801f/volumes/kubernetes.io~projected/kube-api-access-4xzg5 major:0 minor:64 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/55654674c6da37dc7ece64febaac9f1a0c90fae45d1b6ea63d40b29d557e5364/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/a4830250f1a5d4728e20229932e3c2036ccffff068987dbcee6a580546870798/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/a474a5fd2707137ec96379fa4e662b1097419bfe78683ddeef08f52c146d32ce/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/48dd1f9230e33c89d96b3fb4f97187aa358cc6416005201f532c70d58ca58b25/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/d27bc9252ac20baa2a29bd5be63f3f3d2d32f5af9ff947d6eb0945d3a3177714/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/91461be0de6780c76c4ac186d8f6f153651605355704a2d027773f7146f61615/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/b565f5562ad3f49a6c803131a59cc3b30f33644760cd71b155ebac70310e0e4c/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/df20d13f00af65084eba7c4a55e763790ffcc232653a6c400adc241a11c04f0d/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/2ac31f259990a0c1746578e26f4acfd46b11c6d9b344373f3255ea7909914b93/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-133:{mountpoint:/var/lib/containers/storage/overlay/1d4bf9ecbdea3444d414e253a65b6635174d9a312642eda1f8bb7b0b4650fbce/merged major:0 minor:133 fsType:overlay blockSize:0} overlay_0-138:{mountpoint:/var/lib/containers/storage/overlay/435ac83d653964f3f8ec32dc9a45bb4085cf4d77e996cea597efddf035034da8/merged major:0 minor:138 fsType:overlay blockSize:0} overlay_0-140:{mountpoint:/var/lib/containers/storage/overlay/ecf8c8d4d6727fbd83f0abcc1e99eea2a2ca4aaec5aefda2db3d3c8c4633ce82/merged major:0 minor:140 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/cd1fefd53e80c132cfcf7d4eaa2ca74d6817db5f4bf4af05f8a26e67abe9457f/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/31c8a24196b8e7a987928136942d489ef86fae0e24dd119b35d50dcb0ecb4c22/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/ad5085f85a88709fde3da4d09ece1f27f395c99a62c64fc9e08e4f70eb993d1b/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/1d77bf5e5ced8b4385d1ae8cd46e51d0dd795af796b331e844e792970c8f37a9/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/ea33ba4e8c89bb7dc4160671adb81fe4fafc9c1c8e580d83767da4d0d38f2a30/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/654d02134ce8b653051900c1101f3fa6e3f692527c31604fd046e2307ea97245/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/47c4c8af7eb9d68bc44dfce7c4053655c789e29d9467fc0107b4d94a4925cfdb/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/56fbabe116313c6a8d034335a4e73714642defe1881e21cf57830e35f4564fba/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/582d6596fd4e818f3ad06b50cfb4fbbc00607beefd99004b5ba36fd7002e4d89/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/5f82c59a0a18d99b8eb43a1f4988eda5ce9cf4b536652dfffed53786d1d97618/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-206:{mountpoint:/var/lib/containers/storage/overlay/15472239f63dbf7b13467789fb4448755451c5dc8921587bf9cf4e52018cc73e/merged major:0 minor:206 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/c680937dbe7fbf4ab416a4bd45f40dd7730af27ddf44aef1f61abd7d58425162/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/be22579216004f4a6401d6620198148ab5af53ae7591947e476ab0947d85ccff/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-227:{mountpoint:/var/lib/containers/storage/overlay/3b43d05f440d40c4723a2f43631cd2bb83ec6ef66c21f9236a2c30d11a5a4732/merged major:0 minor:227 fsType:overlay blockSize:0} overlay_0-235:{mountpoint:/var/lib/containers/storage/overlay/0d3e00e637d40ac4096b76243db76928b5591e841f14289dbf60540d97c376e0/merged major:0 minor:235 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/fc6207e31afa8af9184cb1b47a8c663252c565cd348f5dc82b911fa445067f7e/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-239:{mountpoint:/var/lib/containers/storage/overlay/af5d6e551262a0ff19dc8ee90ae7be9689004a47d44f8126df02d454f7f8f633/merged major:0 minor:239 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/cf8d8ae6cab99e2c7462fe54be7ff5e6e03f828775fcd42de046fadda3c1d91a/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/51eb7d9b089c2117d257cad3da8fc63427a814cc2018d063164d9dbb9b41cdfc/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/a85ebc5967b83e8a858993e1ff43c5a6675dcd40c612bf53a4e7dbc3686fc6cf/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/8b10bb400099602fc44332068618b80d80db524aac60793b38829160e5c3b04a/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/e28489c7719023f5ac45a81d559d6caf73debfca9a1c6fda62837eed2431cbb6/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/a2107db8a25aacc8e72c499e20f900fd87d20f6e87847448014863a6d5c252cf/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/8dd6d6f61037e0ba40f820437d33465803711ce478e5e3b74b3c95eab2e9a3f7/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/ace3c0487b3f5106ee48b975e12a89f370dfa5740601ed3959bf03ba6881b801/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/ced09a9389c01624050cda05d32af8dbffe0730340460f96ff9098eef4f645e3/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/6e5b1f5b2a4f1559420ce014d2497a92c33e552c4be5a8b8f8662561da55dc0b/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/cf61911e0698a3200bcd291126caa9bf27d5378a3bf28c26a1b490dd1aa96914/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/4f60514279e9fb9bc43e03d4e901bf98fadab0a064c7971b33c89df69ad2efa8/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/ae29ab6180f13fa0f39b3cd5b147d8dca63eed440c2e62dc6c025b1ba0ed67a2/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/2e04e218f6f47c5890f0e1f0faf086b2f23b52519896caea358d1bb3000650cf/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 31 07:29:18.823796 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.823432    4322 manager.go:217] Machine: {Timestamp:2026-10-31 07:29:18.822929721 +0000 UTC m=+0.086750323 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103595 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c87ff65477149382115d754641c0d SystemUUID:ec2c87ff-6547-7149-3821-15d754641c0d BootID:5bbc108c-7a56-46dc-9d11-43297cde19dc Filesystems:[{Device:/var/lib/kubelet/pods/881291b2-1e3f-491d-8dd8-b1500fc5ab24/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:58 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/374fae1c-44be-4729-802f-e190816bf249/volumes/kubernetes.io~projected/kube-api-access-8vt2p DeviceMajor:0 DeviceMinor:61 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/05163761-47bd-4b1b-93fc-1ef20691a84f/volumes/kubernetes.io~projected/kube-api-access-vsbrt DeviceMajor:0 DeviceMinor:201 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2cce4a82ce2acb5cc08f94741880dd8593b5ea7b5337e8cec54651c480361395/userdata/shm DeviceMajor:0 DeviceMinor:218 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/daacb703ff4a2581c23b6baa731c6c0428df334169ba8f0b6a522387e75f9e71/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d79ad42faa964e86d26e4e258cbdc586d82e52d412b60be7136c10b74449a743/userdata/shm DeviceMajor:0 DeviceMinor:215 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/1a089cc1-9715-4809-9479-02fddca15479/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:56 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/e07914e4-a5e8-42b8-889a-5fab59642856/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:57 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a089cc1-9715-4809-9479-02fddca15479/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/72245812-04f1-47b8-a22f-67a61f175b63/volumes/kubernetes.io~projected/kube-api-access-2978v DeviceMajor:0 DeviceMinor:213 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-227 DeviceMajor:0 DeviceMinor:227 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e9840048-339a-4d1a-a37c-cdfad40f8883/volumes/kubernetes.io~projected/kube-api-access-sl7mm DeviceMajor:0 DeviceMinor:230 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/b1ca5663-7cde-4b55-b971-e2afaf82fb5f/volumes/kubernetes.io~projected/kube-api-access-699ww DeviceMajor:0 DeviceMinor:62 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/ed269b7c-e48c-4732-acc4-c0250a95801f/volumes/kubernetes.io~projected/kube-api-access-4xzg5 DeviceMajor:0 DeviceMinor:64 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/d95920d0-5a98-46bc-9b64-f820f02d5b1d/volumes/kubernetes.io~projected/kube-api-access-n4s6h DeviceMajor:0 DeviceMinor:65 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/deac7042-7310-4a66-98b3-88fbebb4f2f9/volumes/kubernetes.io~projected/kube-api-access-pcn6k DeviceMajor:0 DeviceMinor:217 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/1a089cc1-9715-4809-9479-02fddca15479/volumes/kubernetes.io~projected/kube-api-access-xtwtf DeviceMajor:0 DeviceMinor:66 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/05163761-47bd-4b1b-93fc-1ef20691a84f/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/8f393f8a-63f4-41d7-aedd-d3df202eff35/volumes/kubernetes.io~projected/kube-api-access-zsjgx DeviceMajor:0 DeviceMinor:214 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/04a7d71a2f31142041e2647831fe00ffed120ad1b4103003e326b8698aee6fa0/userdata/shm DeviceMajor:0 DeviceMinor:220 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-235 DeviceMajor:0 DeviceMinor:235 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bf9240bcbb27028905a2df9dc22ad44f2f83a0db479f52303bd11a643739d80a/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e9840048-339a-4d1a-a37c-cdfad40f8883/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:229 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/881291b2-1e3f-491d-8dd8-b1500fc5ab24/volumes/kubernetes.io~projected/kube-api-access-kbq66 DeviceMajor:0 DeviceMinor:59 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6880eb0e8fe90c744e2d7d2097a3361b4478005f3e29885f235f64bdc4ca5303/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/05020db1dc97a26ed98891970b362b7028cae202d49b24ab7904f8897703477c/userdata/shm DeviceMajor:0 DeviceMinor:231 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1d4331e6aaff22c7074c470a5976c2336213eae4059b468255e4282c2bc9d3de/userdata/shm DeviceMajor:0 DeviceMinor:233 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-140 DeviceMajor:0 DeviceMinor:140 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9ae7f604-ca2d-4ddd-8aac-3f3975ef7463/volumes/kubernetes.io~projected/kube-api-access-tcsmv DeviceMajor:0 DeviceMinor:224 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1306f68e8130e53948cb2eb356fd5d7930dac4bbc36095df328e7375bc5ffdea/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-133 DeviceMajor:0 DeviceMinor:133 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/62191e70-8059-4950-b636-b6c58d33b128/volumes/kubernetes.io~projected/kube-api-access-979pp DeviceMajor:0 DeviceMinor:60 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8c73287f7d0aa2a7cbd56a488b7d4e9cbd5dfd0549c78c5610c4d4581d8e89e0/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/828e47b40abf587bf9f22ade36b0923d9911ce9d6a1641a9af54fe8f7ae953f8/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-138 DeviceMajor:0 DeviceMinor:138 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/83315d191c3d017dcefd0d2fe3ee667ae8e2d9d62dacc42a89fedbb045e39319/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/35807caf7071ae80511bf71e7553e3f12001be5ddf3f0ae1c51c87e5fbacbb16/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5be6a30e6055de07f050a907898633d0f0bfe587d1686f45171dae6330c8476d/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/881291b2-1e3f-491d-8dd8-b1500fc5ab24/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/11dc8b0e-286d-42e1-bfe0-d882e7dfe073/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:208 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6fff91a9f5dad61d1b0a19109961e99874b1dd5f8faa562d54b9cb23426ae956/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/cf2415aa-becd-409e-bb51-05d94a380deb/volumes/kubernetes.io~projected/kube-api-access-5624d DeviceMajor:0 DeviceMinor:63 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/0f24a74d-99e3-4bce-a9e1-2eb550655093/volumes/kubernetes.io~projected/kube-api-access-75t9x DeviceMajor:0 DeviceMinor:67 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9ace0b66dfed7d90a6eafcab8506be3c744bf165e2104be4070aa7e2cf55744a/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/b638fc1f-3579-467c-ac54-e4a4bc891f9e/volumes/kubernetes.io~projected/kube-api-access-vfsf9 DeviceMajor:0 DeviceMinor:197 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-206 DeviceMajor:0 DeviceMinor:206 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-239 DeviceMajor:0 DeviceMinor:239 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:04a7d71a2f31142 MacAddress:aa:c6:f1:f2:fc:22 Speed:10000 Mtu:8901} {Name:05020db1dc97a26 MacAddress:b6:cf:b6:e6:3b:f3 Speed:10000 Mtu:8901} {Name:1d4331e6aaff22c MacAddress:16:ac:6f:77:47:5c Speed:10000 Mtu:8901} {Name:2cce4a82ce2acb5 MacAddress:b2:00:2e:de:21:8e Speed:10000 Mtu:8901} {Name:5be6a30e6055de0 MacAddress:46:a5:f6:56:53:dc Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:f7:04:22:61 Speed:0 Mtu:9001} {Name:br-int MacAddress:9e:fd:62:9f:74:a3 Speed:0 Mtu:8901} {Name:d79ad42faa964e8 MacAddress:72:f0:0d:81:ff:e5 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:f7:04:22:61 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:ba:fb:b7:50:cb:26 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:a2:52:2b:e0:00:a6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 07:29:18.823796 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.823714    4322 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 07:29:18.823958 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.823812    4322 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 07:29:18.824280 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.824256    4322 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 07:29:18.824439 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.824281    4322 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-234.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 07:29:18.824481 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.824442    4322 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 07:29:18.824481 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.824450    4322 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 07:29:18.824481 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.824465    4322 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 07:29:18.824481 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.824478    4322 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 07:29:18.824588 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.824547    4322 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 07:29:18.824701 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.824692    4322 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 07:29:18.824953 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.824943    4322 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 07:29:18.824989 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.824981    4322 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 07:29:18.825019 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.824998    4322 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 07:29:18.825019 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.825009    4322 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 07:29:18.825019 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.825018    4322 kubelet.go:405] "Adding apiserver pod source"
Oct 31 07:29:18.825115 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.825025    4322 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 07:29:18.825573 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.825557    4322 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 07:29:18.825573 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.825569    4322 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 07:29:18.825984 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.825965    4322 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 07:29:18.827931 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.827919    4322 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 07:29:18.827931 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.827932    4322 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 07:29:18.828030 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828020    4322 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 31 07:29:18.828196 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828186    4322 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 07:29:18.828240 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828214    4322 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 07:29:18.828331 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828321    4322 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 07:29:18.828383 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828337    4322 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 07:29:18.828383 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828347    4322 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 07:29:18.828383 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828355    4322 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 07:29:18.828383 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828363    4322 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 07:29:18.828383 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828377    4322 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 07:29:18.828383 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828384    4322 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 07:29:18.828605 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828390    4322 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 07:29:18.828605 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828397    4322 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 07:29:18.828605 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828404    4322 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 07:29:18.828605 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828420    4322 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 07:29:18.828605 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828429    4322 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 07:29:18.828605 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828452    4322 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 07:29:18.828605 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.828458    4322 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 07:29:18.831784 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.831771    4322 server.go:1263] "Started kubelet"
Oct 31 07:29:18.831866 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.831837    4322 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 07:29:18.831927 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.831865    4322 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 07:29:18.832038 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.832025    4322 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 07:29:18.832332 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.832308    4322 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 07:29:18.832478 ip-10-0-138-234 systemd[1]: Started Kubernetes Kubelet.
Oct 31 07:29:18.833292 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.833276    4322 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 07:29:18.836941 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.836922    4322 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 07:29:18.836941 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.836931    4322 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 07:29:18.837096 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.837008    4322 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 07:23:40 +0000 UTC" deadline="2028-04-27 13:12:43.125490169 +0000 UTC"
Oct 31 07:29:18.837096 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.837046    4322 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13061h43m24.288447088s"
Oct 31 07:29:18.837096 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.837080    4322 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 07:29:18.837096 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.837086    4322 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 07:29:18.837307 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:18.837095    4322 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-234.ec2.internal\" not found"
Oct 31 07:29:18.837307 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.837179    4322 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 07:29:18.839139 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.838807    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e07914e4-a5e8-42b8-889a-5fab59642856" volumeName="kubernetes.io/secret/e07914e4-a5e8-42b8-889a-5fab59642856-agent-certs" seLinuxMountContext=""
Oct 31 07:29:18.839288 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839148    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11dc8b0e-286d-42e1-bfe0-d882e7dfe073" volumeName="kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret" seLinuxMountContext=""
Oct 31 07:29:18.839288 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839162    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="374fae1c-44be-4729-802f-e190816bf249" volumeName="kubernetes.io/projected/374fae1c-44be-4729-802f-e190816bf249-kube-api-access-8vt2p" seLinuxMountContext=""
Oct 31 07:29:18.839288 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839170    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b638fc1f-3579-467c-ac54-e4a4bc891f9e" volumeName="kubernetes.io/configmap/b638fc1f-3579-467c-ac54-e4a4bc891f9e-config-volume" seLinuxMountContext=""
Oct 31 07:29:18.839288 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839179    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf2415aa-becd-409e-bb51-05d94a380deb" volumeName="kubernetes.io/configmap/cf2415aa-becd-409e-bb51-05d94a380deb-iptables-alerter-script" seLinuxMountContext=""
Oct 31 07:29:18.839288 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839193    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9840048-339a-4d1a-a37c-cdfad40f8883" volumeName="kubernetes.io/projected/e9840048-339a-4d1a-a37c-cdfad40f8883-kube-api-access-sl7mm" seLinuxMountContext=""
Oct 31 07:29:18.839288 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839207    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0f24a74d-99e3-4bce-a9e1-2eb550655093" volumeName="kubernetes.io/configmap/0f24a74d-99e3-4bce-a9e1-2eb550655093-cni-binary-copy" seLinuxMountContext=""
Oct 31 07:29:18.839288 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839215    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0f24a74d-99e3-4bce-a9e1-2eb550655093" volumeName="kubernetes.io/configmap/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-daemon-config" seLinuxMountContext=""
Oct 31 07:29:18.839288 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839240    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a089cc1-9715-4809-9479-02fddca15479" volumeName="kubernetes.io/projected/1a089cc1-9715-4809-9479-02fddca15479-kube-api-access-xtwtf" seLinuxMountContext=""
Oct 31 07:29:18.839288 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839272    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62191e70-8059-4950-b636-b6c58d33b128" volumeName="kubernetes.io/projected/62191e70-8059-4950-b636-b6c58d33b128-kube-api-access-979pp" seLinuxMountContext=""
Oct 31 07:29:18.839288 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839289    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f393f8a-63f4-41d7-aedd-d3df202eff35" volumeName="kubernetes.io/empty-dir/8f393f8a-63f4-41d7-aedd-d3df202eff35-data-volume" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839298    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f393f8a-63f4-41d7-aedd-d3df202eff35" volumeName="kubernetes.io/projected/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-api-access-zsjgx" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839306    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ae7f604-ca2d-4ddd-8aac-3f3975ef7463" volumeName="kubernetes.io/projected/9ae7f604-ca2d-4ddd-8aac-3f3975ef7463-kube-api-access-tcsmv" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839313    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf2415aa-becd-409e-bb51-05d94a380deb" volumeName="kubernetes.io/projected/cf2415aa-becd-409e-bb51-05d94a380deb-kube-api-access-5624d" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839323    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="881291b2-1e3f-491d-8dd8-b1500fc5ab24" volumeName="kubernetes.io/projected/881291b2-1e3f-491d-8dd8-b1500fc5ab24-kube-api-access-kbq66" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839330    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1ca5663-7cde-4b55-b971-e2afaf82fb5f" volumeName="kubernetes.io/projected/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-kube-api-access-699ww" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839339    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="deac7042-7310-4a66-98b3-88fbebb4f2f9" volumeName="kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-trusted-ca-bundle" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839348    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9840048-339a-4d1a-a37c-cdfad40f8883" volumeName="kubernetes.io/configmap/e9840048-339a-4d1a-a37c-cdfad40f8883-signing-cabundle" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839356    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9840048-339a-4d1a-a37c-cdfad40f8883" volumeName="kubernetes.io/secret/e9840048-339a-4d1a-a37c-cdfad40f8883-signing-key" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839367    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed269b7c-e48c-4732-acc4-c0250a95801f" volumeName="kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839375    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed269b7c-e48c-4732-acc4-c0250a95801f" volumeName="kubernetes.io/projected/ed269b7c-e48c-4732-acc4-c0250a95801f-kube-api-access-4xzg5" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839385    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05163761-47bd-4b1b-93fc-1ef20691a84f" volumeName="kubernetes.io/secret/05163761-47bd-4b1b-93fc-1ef20691a84f-cert" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839396    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a089cc1-9715-4809-9479-02fddca15479" volumeName="kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-ovnkube-config" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839410    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1ca5663-7cde-4b55-b971-e2afaf82fb5f" volumeName="kubernetes.io/empty-dir/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-tmp-dir" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839421    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="deac7042-7310-4a66-98b3-88fbebb4f2f9" volumeName="kubernetes.io/empty-dir/deac7042-7310-4a66-98b3-88fbebb4f2f9-archives-path" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839429    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed269b7c-e48c-4732-acc4-c0250a95801f" volumeName="kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839438    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05163761-47bd-4b1b-93fc-1ef20691a84f" volumeName="kubernetes.io/projected/05163761-47bd-4b1b-93fc-1ef20691a84f-kube-api-access-vsbrt" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839471    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a089cc1-9715-4809-9479-02fddca15479" volumeName="kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-env-overrides" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839489    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a089cc1-9715-4809-9479-02fddca15479" volumeName="kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-ovnkube-script-lib" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839500    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a089cc1-9715-4809-9479-02fddca15479" volumeName="kubernetes.io/secret/1a089cc1-9715-4809-9479-02fddca15479-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839507    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d95920d0-5a98-46bc-9b64-f820f02d5b1d" volumeName="kubernetes.io/projected/d95920d0-5a98-46bc-9b64-f820f02d5b1d-kube-api-access-n4s6h" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839514    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="deac7042-7310-4a66-98b3-88fbebb4f2f9" volumeName="kubernetes.io/projected/deac7042-7310-4a66-98b3-88fbebb4f2f9-kube-api-access-pcn6k" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839522    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62191e70-8059-4950-b636-b6c58d33b128" volumeName="kubernetes.io/configmap/62191e70-8059-4950-b636-b6c58d33b128-serviceca" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839529    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="881291b2-1e3f-491d-8dd8-b1500fc5ab24" volumeName="kubernetes.io/empty-dir/881291b2-1e3f-491d-8dd8-b1500fc5ab24-tmp" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839536    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="881291b2-1e3f-491d-8dd8-b1500fc5ab24" volumeName="kubernetes.io/empty-dir/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-tuned" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839552    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b638fc1f-3579-467c-ac54-e4a4bc891f9e" volumeName="kubernetes.io/empty-dir/b638fc1f-3579-467c-ac54-e4a4bc891f9e-tmp-dir" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839560    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="deac7042-7310-4a66-98b3-88fbebb4f2f9" volumeName="kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-service-ca-bundle" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839568    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e07914e4-a5e8-42b8-889a-5fab59642856" volumeName="kubernetes.io/configmap/e07914e4-a5e8-42b8-889a-5fab59642856-konnectivity-ca" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839578    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72245812-04f1-47b8-a22f-67a61f175b63" volumeName="kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839589    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b638fc1f-3579-467c-ac54-e4a4bc891f9e" volumeName="kubernetes.io/projected/b638fc1f-3579-467c-ac54-e4a4bc891f9e-kube-api-access-vfsf9" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839601    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed269b7c-e48c-4732-acc4-c0250a95801f" volumeName="kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-cni-binary-copy" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839614    4322 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0f24a74d-99e3-4bce-a9e1-2eb550655093" volumeName="kubernetes.io/projected/0f24a74d-99e3-4bce-a9e1-2eb550655093-kube-api-access-75t9x" seLinuxMountContext=""
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839623    4322 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 07:29:18.839768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.839629    4322 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 07:29:18.841355 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.841331    4322 factory.go:55] Registering systemd factory
Oct 31 07:29:18.841355 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.841354    4322 factory.go:223] Registration of the systemd container factory successfully
Oct 31 07:29:18.841584 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.841565    4322 factory.go:153] Registering CRI-O factory
Oct 31 07:29:18.841584 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.841586    4322 factory.go:223] Registration of the crio container factory successfully
Oct 31 07:29:18.841717 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.841640    4322 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 07:29:18.841717 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.841664    4322 factory.go:103] Registering Raw factory
Oct 31 07:29:18.841717 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.841677    4322 manager.go:1199] Started watching for new ooms in manager
Oct 31 07:29:18.841948 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.841936    4322 manager.go:319] Starting recovery of all containers
Oct 31 07:29:18.863397 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.863299    4322 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 07:29:18.864816 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.864787    4322 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 07:29:18.864816 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.864814    4322 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 07:29:18.864982 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.864836    4322 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 07:29:18.864982 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:18.864878    4322 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 31 07:29:18.888559 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.888531    4322 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:29:18.888712 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.888578    4322 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 31 07:29:18.889880 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.889856    4322 generic.go:356] "Generic (PLEG): container finished" podID="ed269b7c-e48c-4732-acc4-c0250a95801f" containerID="c7ba45d3b9a3444f3cef07e45b0da9950050f0d007875035296e1f590fecf11c" exitCode=0
Oct 31 07:29:18.889880 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.889878    4322 generic.go:356] "Generic (PLEG): container finished" podID="ed269b7c-e48c-4732-acc4-c0250a95801f" containerID="d97137795423119cad0214038fa74f2bcb1c1bb2a7804471a045f6c74defafa6" exitCode=0
Oct 31 07:29:18.890043 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.889887    4322 generic.go:356] "Generic (PLEG): container finished" podID="ed269b7c-e48c-4732-acc4-c0250a95801f" containerID="62aae0bcd549349da519b346b911df76183d3d297fa3ba7f61576ba2fb80c903" exitCode=0
Oct 31 07:29:18.890043 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.889895    4322 generic.go:356] "Generic (PLEG): container finished" podID="ed269b7c-e48c-4732-acc4-c0250a95801f" containerID="ce18a6398904b5dcfb31552c6187ba05339d337ae60a062d24ce0befdb778b0b" exitCode=0
Oct 31 07:29:18.890043 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.889902    4322 generic.go:356] "Generic (PLEG): container finished" podID="ed269b7c-e48c-4732-acc4-c0250a95801f" containerID="3afe7ff113255d0cda263f4c817a20ece5475cb12253dd90bf505a2fd2de6446" exitCode=0
Oct 31 07:29:18.890043 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.889908    4322 generic.go:356] "Generic (PLEG): container finished" podID="ed269b7c-e48c-4732-acc4-c0250a95801f" containerID="0b435dd8d7cd695a80ae77537cf8c506530e93ccebe8e12a1cf482bb18c50bc1" exitCode=0
Oct 31 07:29:18.897393 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.897366    4322 generic.go:356] "Generic (PLEG): container finished" podID="ee833da619c66751ed2de8e982aff1e5" containerID="45347de82f09b36205869e5d68561611d0a44af1823936038adeaedcbb4121fd" exitCode=0
Oct 31 07:29:18.898821 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.898800    4322 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:29:18.901580 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.901561    4322 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:29:18.901737 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.901722    4322 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:29:18.916785 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.916764    4322 manager.go:324] Recovery completed
Oct 31 07:29:18.933717 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.933665    4322 cpu_manager.go:226] "Starting" policy="none"
Oct 31 07:29:18.933717 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.933679    4322 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 07:29:18.933717 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.933694    4322 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 07:29:18.933854 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.933827    4322 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 31 07:29:18.934253 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.933838    4322 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 31 07:29:18.934285 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.934256    4322 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 31 07:29:18.934285 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.934262    4322 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 31 07:29:18.934285 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.934268    4322 policy_none.go:50] "Start"
Oct 31 07:29:18.934285 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.934277    4322 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 07:29:18.934390 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.934288    4322 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 07:29:18.934421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.934416    4322 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 31 07:29:18.934449 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.934423    4322 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 31 07:29:18.934449 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.934429    4322 policy_none.go:44] "Start"
Oct 31 07:29:18.939194 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.939168    4322 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 07:29:18.941662 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.941646    4322 manager.go:341] "Starting Device Plugin manager"
Oct 31 07:29:18.941749 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:18.941668    4322 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 07:29:18.941749 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.941676    4322 server.go:87] "Starting device plugin registration server"
Oct 31 07:29:18.941990 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.941977    4322 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 07:29:18.942025 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.941991    4322 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 07:29:18.942083 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.942068    4322 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 07:29:18.942195 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.942161    4322 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 07:29:18.942255 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.942204    4322 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 07:29:18.942255 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.942211    4322 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 07:29:18.942343 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.942280    4322 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T07:29:18.94219402Z","UUID":"a7fbece7-5667-40b0-868a-418b97c14c11","Handler":null,"Name":"","Endpoint":""}
Oct 31 07:29:18.943420 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:18.943400    4322 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 07:29:18.943717 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.943696    4322 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 07:29:18.943796 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.943725    4322 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 07:29:18.964977 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.964939    4322 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal"]
Oct 31 07:29:18.965520 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.965479    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal" event={"ID":"700ea6542926bcf6606c986e3906f152","Type":"ContainerStarted","Data":"6f1dfe8c0ef1b1d2af64e7e7709829070dc213e5ed7b3f10e931c4c13b31fc5a"}
Oct 31 07:29:18.965604 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.965525    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal" event={"ID":"700ea6542926bcf6606c986e3906f152","Type":"ContainerStarted","Data":"6fff91a9f5dad61d1b0a19109961e99874b1dd5f8faa562d54b9cb23426ae956"}
Oct 31 07:29:18.965604 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.965542    4322 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:18.965604 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.965549    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" event={"ID":"ee833da619c66751ed2de8e982aff1e5","Type":"ContainerStarted","Data":"bdc45b5dd4a27256b57033750a80a9d6411e41d9ccc94366d1927e6cd02406ed"}
Oct 31 07:29:18.965604 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.965569    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" event={"ID":"ee833da619c66751ed2de8e982aff1e5","Type":"ContainerDied","Data":"45347de82f09b36205869e5d68561611d0a44af1823936038adeaedcbb4121fd"}
Oct 31 07:29:18.965604 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.965592    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal" event={"ID":"ee833da619c66751ed2de8e982aff1e5","Type":"ContainerStarted","Data":"bf9240bcbb27028905a2df9dc22ad44f2f83a0db479f52303bd11a643739d80a"}
Oct 31 07:29:18.965604 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.965598    4322 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:18.986447 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.986423    4322 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 07:29:18.986532 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:18.986460    4322 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-138-234.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:18.986980 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:18.986965    4322 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 07:29:18.987044 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:18.987003    4322 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.044315 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.044291    4322 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:29:19.045577 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.045558    4322 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 07:29:19.045670 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.045589    4322 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 07:29:19.045670 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.045600    4322 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-234.ec2.internal" event="NodeHasSufficientPID"
Oct 31 07:29:19.045670 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.045634    4322 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.081340 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.081323    4322 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.081411 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.081370    4322 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.089927 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.089903    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee833da619c66751ed2de8e982aff1e5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"ee833da619c66751ed2de8e982aff1e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.089996 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.089928    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee833da619c66751ed2de8e982aff1e5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"ee833da619c66751ed2de8e982aff1e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.089996 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.089943    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/700ea6542926bcf6606c986e3906f152-config\") pod \"kube-apiserver-proxy-ip-10-0-138-234.ec2.internal\" (UID: \"700ea6542926bcf6606c986e3906f152\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.190729 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.190670    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee833da619c66751ed2de8e982aff1e5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"ee833da619c66751ed2de8e982aff1e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.190729 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.190705    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee833da619c66751ed2de8e982aff1e5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"ee833da619c66751ed2de8e982aff1e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.190843 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.190744    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/700ea6542926bcf6606c986e3906f152-config\") pod \"kube-apiserver-proxy-ip-10-0-138-234.ec2.internal\" (UID: \"700ea6542926bcf6606c986e3906f152\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.190843 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.190760    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee833da619c66751ed2de8e982aff1e5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"ee833da619c66751ed2de8e982aff1e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.190843 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.190794    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee833da619c66751ed2de8e982aff1e5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" (UID: \"ee833da619c66751ed2de8e982aff1e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.190843 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.190834    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/700ea6542926bcf6606c986e3906f152-config\") pod \"kube-apiserver-proxy-ip-10-0-138-234.ec2.internal\" (UID: \"700ea6542926bcf6606c986e3906f152\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.825732 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.825700    4322 apiserver.go:52] "Watching apiserver"
Oct 31 07:29:19.879858 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.879794    4322 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 07:29:19.880261 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.880218    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-z22fp","openshift-dns/dns-default-w78n9","openshift-multus/multus-additional-cni-plugins-h4qfb","openshift-image-registry/node-ca-fl4w7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal","kube-system/konnectivity-agent-6ll6x","openshift-ingress-canary/ingress-canary-vktw2","openshift-insights/periodic-gathering-k76m7-4srkl","kube-system/global-pull-secret-syncer-z9h8d","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l","openshift-multus/multus-jghjq","openshift-service-ca/service-ca-5fffc97bdf-zn7pt","kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal","openshift-multus/network-metrics-daemon-2skf7","openshift-cluster-node-tuning-operator/tuned-m7rmk","openshift-ovn-kubernetes/ovnkube-node-vzgl4","openshift-insights/insights-runtime-extractor-wqkwf","openshift-kube-storage-version-migrator/migrator-66df5d496b-s2zkk","openshift-dns/node-resolver-gkl8q","openshift-network-diagnostics/network-check-target-pjddt"]
Oct 31 07:29:19.881014 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.880998    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:29:19.885205 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.885183    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:19.888959 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.888936    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:19.895573 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.895556    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w4rqx\""
Oct 31 07:29:19.895664 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.895583    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-qvcsq\""
Oct 31 07:29:19.895726 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.895711    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rhfwc\""
Oct 31 07:29:19.895782 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.895771    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 07:29:19.896286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.896273    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-krpl2\""
Oct 31 07:29:19.896530 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.896503    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-lzwlj\""
Oct 31 07:29:19.896530 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.896506    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 07:29:19.896679 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.896571    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-ddkhs\""
Oct 31 07:29:19.896679 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.896662    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9v74w\""
Oct 31 07:29:19.897055 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.897037    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 07:29:19.897131 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.897037    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 07:29:19.897131 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.897042    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 07:29:19.897131 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.897088    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 07:29:19.897131 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.897092    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 07:29:19.897131 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.897088    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 07:29:19.897131 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.897119    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 07:29:19.897131 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.897125    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 07:29:19.897131 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.897096    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 07:29:19.897433 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.897087    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 07:29:19.897433 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.897168    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 07:29:19.897433 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.897183    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 07:29:19.897433 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.897200    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 07:29:19.906404 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.906386    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 07:29:19.906558 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.906545    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 07:29:19.906618 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.906592    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 07:29:19.906669 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.906627    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 07:29:19.906669 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.906647    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 07:29:19.906747 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.906653    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 07:29:19.906747 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.906664    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 07:29:19.906747 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.906733    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 07:29:19.906942 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.906929    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 07:29:19.907920 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.907906    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 07:29:19.907988 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.907975    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-tdhmc\""
Oct 31 07:29:19.908179 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.908164    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 31 07:29:19.908288 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.908197    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 07:29:19.908981 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.908961    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 07:29:19.908981 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.908974    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 07:29:19.909090 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909002    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 07:29:19.909090 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909020    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 07:29:19.909090 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909028    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 31 07:29:19.909090 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909033    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 31 07:29:19.909090 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909088    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 07:29:19.909297 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.908972    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hr94j\""
Oct 31 07:29:19.909380 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909294    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 31 07:29:19.909380 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909295    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 31 07:29:19.909533 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909308    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 07:29:19.909533 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909310    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 07:29:19.909533 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909324    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 31 07:29:19.909533 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909329    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 31 07:29:19.909533 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909424    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 07:29:19.909533 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909333    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dvd7d\""
Oct 31 07:29:19.909533 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909332    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 07:29:19.909533 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909345    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 07:29:19.909533 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909389    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 07:29:19.909533 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909329    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 07:29:19.909533 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909529    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-dl9sg\""
Oct 31 07:29:19.909533 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909527    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-9bh97\""
Oct 31 07:29:19.909909 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909571    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-cp4x6\""
Oct 31 07:29:19.909909 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909574    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 07:29:19.909909 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909581    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mwmjb\""
Oct 31 07:29:19.909909 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909590    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-48xxx\""
Oct 31 07:29:19.909909 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909596    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-gt84m\""
Oct 31 07:29:19.909909 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909604    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 07:29:19.909909 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909652    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xcjxx\""
Oct 31 07:29:19.909909 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.909838    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mrcbt\""
Oct 31 07:29:19.913111 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.913093    4322 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.913218 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.913202    4322 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.919046 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.919027    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 31 07:29:19.936033 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.936015    4322 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 07:29:19.936093 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:19.936056    4322 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-138-234.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.936470 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.936456    4322 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 07:29:19.936514 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:19.936491    4322 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-234.ec2.internal"
Oct 31 07:29:19.938426 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.938414    4322 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 07:29:19.994702 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.994681    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.994783 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.994709    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbq66\" (UniqueName: \"kubernetes.io/projected/881291b2-1e3f-491d-8dd8-b1500fc5ab24-kube-api-access-kbq66\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.994783 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.994725    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-dbus\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:19.994783 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.994739    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:19.994899 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.994799    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f393f8a-63f4-41d7-aedd-d3df202eff35-data-volume\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:19.994899 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.994870    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f393f8a-63f4-41d7-aedd-d3df202eff35-data-volume\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:19.995070 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.994901    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-cni-bin\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.995070 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.994938    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xzg5\" (UniqueName: \"kubernetes.io/projected/ed269b7c-e48c-4732-acc4-c0250a95801f-kube-api-access-4xzg5\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:19.995070 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.994965    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/deac7042-7310-4a66-98b3-88fbebb4f2f9-archives-path\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:19.995246 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995099    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/deac7042-7310-4a66-98b3-88fbebb4f2f9-archives-path\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:19.995437 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995423    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-ovnkube-script-lib\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.995500 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995451    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:19.995500 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995479    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-tuned\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.995593 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995507    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/881291b2-1e3f-491d-8dd8-b1500fc5ab24-tmp\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.995593 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995532    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcsmv\" (UniqueName: \"kubernetes.io/projected/9ae7f604-ca2d-4ddd-8aac-3f3975ef7463-kube-api-access-tcsmv\") pod \"migrator-66df5d496b-s2zkk\" (UID: \"9ae7f604-ca2d-4ddd-8aac-3f3975ef7463\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-s2zkk"
Oct 31 07:29:19.995593 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995554    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-ovn\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.995593 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995558    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-original-pull-secret\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:19.995593 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995584    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-tuned\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.995798 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995580    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtwtf\" (UniqueName: \"kubernetes.io/projected/1a089cc1-9715-4809-9479-02fddca15479-kube-api-access-xtwtf\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.995798 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995650    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-modprobe-d\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.995798 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995665    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/881291b2-1e3f-491d-8dd8-b1500fc5ab24-tmp\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.995798 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995697    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-ovnkube-script-lib\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.995798 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995707    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfsf9\" (UniqueName: \"kubernetes.io/projected/b638fc1f-3579-467c-ac54-e4a4bc891f9e-kube-api-access-vfsf9\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:19.995999 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995821    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:19.995999 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995868    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-cnibin\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:19.995999 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995911    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/62191e70-8059-4950-b636-b6c58d33b128-serviceca\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:29:19.995999 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995955    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-systemd-units\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.995999 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.995995    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-etc-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.996164 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996027    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-cni-netd\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.996164 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996048    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysconfig\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.996164 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996063    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-kubelet-config\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:19.996164 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996091    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f24a74d-99e3-4bce-a9e1-2eb550655093-cni-binary-copy\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.996164 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996110    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-conf-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.996339 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996181    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/62191e70-8059-4950-b636-b6c58d33b128-serviceca\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:29:19.996339 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996185    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:19.996339 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996240    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysctl-conf\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.996339 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996270    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-lib-modules\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.996339 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996294    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-k8s-cni-cncf-io\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.996339 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996336    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-etc-kubernetes\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.996570 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996359    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e07914e4-a5e8-42b8-889a-5fab59642856-agent-certs\") pod \"konnectivity-agent-6ll6x\" (UID: \"e07914e4-a5e8-42b8-889a-5fab59642856\") " pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:29:19.996570 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996377    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f24a74d-99e3-4bce-a9e1-2eb550655093-cni-binary-copy\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.996570 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996382    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-699ww\" (UniqueName: \"kubernetes.io/projected/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-kube-api-access-699ww\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:29:19.996570 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996399    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-trusted-ca-bundle\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:19.996570 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996424    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf2415aa-becd-409e-bb51-05d94a380deb-host-slash\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:29:19.996570 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996453    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:19.996570 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996478    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsjgx\" (UniqueName: \"kubernetes.io/projected/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-api-access-zsjgx\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:19.996570 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996499    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-daemon-config\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.996570 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996533    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996573    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf2415aa-becd-409e-bb51-05d94a380deb-iptables-alerter-script\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996622    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-device-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996644    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f393f8a-63f4-41d7-aedd-d3df202eff35-crio-socket\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996693    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e9840048-339a-4d1a-a37c-cdfad40f8883-signing-cabundle\") pod \"service-ca-5fffc97bdf-zn7pt\" (UID: \"e9840048-339a-4d1a-a37c-cdfad40f8883\") " pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996717    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-socket-dir-parent\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996743    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-etc-selinux\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996760    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-daemon-config\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996772    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e07914e4-a5e8-42b8-889a-5fab59642856-agent-certs\") pod \"konnectivity-agent-6ll6x\" (UID: \"e07914e4-a5e8-42b8-889a-5fab59642856\") " pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996768    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-sys-fs\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996799    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-trusted-ca-bundle\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996810    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl7mm\" (UniqueName: \"kubernetes.io/projected/e9840048-339a-4d1a-a37c-cdfad40f8883-kube-api-access-sl7mm\") pod \"service-ca-5fffc97bdf-zn7pt\" (UID: \"e9840048-339a-4d1a-a37c-cdfad40f8883\") " pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996820    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996851    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cf2415aa-becd-409e-bb51-05d94a380deb-iptables-alerter-script\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996908    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1a089cc1-9715-4809-9479-02fddca15479-ovn-node-metrics-cert\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996931    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e9840048-339a-4d1a-a37c-cdfad40f8883-signing-key\") pod \"service-ca-5fffc97bdf-zn7pt\" (UID: \"e9840048-339a-4d1a-a37c-cdfad40f8883\") " pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996960    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-multus-certs\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.996987 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.996977    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-cni-binary-copy\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997021    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsbrt\" (UniqueName: \"kubernetes.io/projected/05163761-47bd-4b1b-93fc-1ef20691a84f-kube-api-access-vsbrt\") pod \"ingress-canary-vktw2\" (UID: \"05163761-47bd-4b1b-93fc-1ef20691a84f\") " pod="openshift-ingress-canary/ingress-canary-vktw2"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997056    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-netns\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997081    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-hostroot\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997084    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e9840048-339a-4d1a-a37c-cdfad40f8883-signing-cabundle\") pod \"service-ca-5fffc97bdf-zn7pt\" (UID: \"e9840048-339a-4d1a-a37c-cdfad40f8883\") " pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997126    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-979pp\" (UniqueName: \"kubernetes.io/projected/62191e70-8059-4950-b636-b6c58d33b128-kube-api-access-979pp\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997178    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e07914e4-a5e8-42b8-889a-5fab59642856-konnectivity-ca\") pod \"konnectivity-agent-6ll6x\" (UID: \"e07914e4-a5e8-42b8-889a-5fab59642856\") " pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997191    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-cni-binary-copy\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997210    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-tmp-dir\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997272    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e9840048-339a-4d1a-a37c-cdfad40f8883-signing-key\") pod \"service-ca-5fffc97bdf-zn7pt\" (UID: \"e9840048-339a-4d1a-a37c-cdfad40f8883\") " pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997319    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-slash\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997338    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-cni-bin\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997363    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-env-overrides\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997388    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-kubernetes\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997449    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1a089cc1-9715-4809-9479-02fddca15479-ovn-node-metrics-cert\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997457    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-tmp-dir\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997513    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e07914e4-a5e8-42b8-889a-5fab59642856-konnectivity-ca\") pod \"konnectivity-agent-6ll6x\" (UID: \"e07914e4-a5e8-42b8-889a-5fab59642856\") " pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997518    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysctl-d\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997516    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-env-overrides\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997572    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:19.997606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997601    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-system-cni-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997643    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5624d\" (UniqueName: \"kubernetes.io/projected/cf2415aa-becd-409e-bb51-05d94a380deb-kube-api-access-5624d\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997664    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-node-log\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997681    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b638fc1f-3579-467c-ac54-e4a4bc891f9e-tmp-dir\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997705    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-cni-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997727    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62191e70-8059-4950-b636-b6c58d33b128-host\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997751    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b638fc1f-3579-467c-ac54-e4a4bc891f9e-tmp-dir\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997749    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-service-ca-bundle\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997786    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcn6k\" (UniqueName: \"kubernetes.io/projected/deac7042-7310-4a66-98b3-88fbebb4f2f9-kube-api-access-pcn6k\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997821    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-run-netns\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997865    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-log-socket\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997883    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-service-ca-bundle\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997895    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-var-lib-kubelet\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997924    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-cnibin\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997949    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-kubelet\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997968    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75t9x\" (UniqueName: \"kubernetes.io/projected/0f24a74d-99e3-4bce-a9e1-2eb550655093-kube-api-access-75t9x\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.997983    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-system-cni-dir\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998020    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05163761-47bd-4b1b-93fc-1ef20691a84f-cert\") pod \"ingress-canary-vktw2\" (UID: \"05163761-47bd-4b1b-93fc-1ef20691a84f\") " pod="openshift-ingress-canary/ingress-canary-vktw2"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998051    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998078    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4s6h\" (UniqueName: \"kubernetes.io/projected/d95920d0-5a98-46bc-9b64-f820f02d5b1d-kube-api-access-n4s6h\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998113    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-kubelet\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998249    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-systemd\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998272    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-socket-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:19.998286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998290    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vt2p\" (UniqueName: \"kubernetes.io/projected/374fae1c-44be-4729-802f-e190816bf249-kube-api-access-8vt2p\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998307    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998314    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05163761-47bd-4b1b-93fc-1ef20691a84f-cert\") pod \"ingress-canary-vktw2\" (UID: \"05163761-47bd-4b1b-93fc-1ef20691a84f\") " pod="openshift-ingress-canary/ingress-canary-vktw2"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998337    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-run-ovn-kubernetes\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998415    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998447    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-run\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998475    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-sys\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998509    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2978v\" (UniqueName: \"kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v\") pod \"network-check-target-pjddt\" (UID: \"72245812-04f1-47b8-a22f-67a61f175b63\") " pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998530    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b638fc1f-3579-467c-ac54-e4a4bc891f9e-config-volume\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998587    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-cni-multus\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998625    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-hosts-file\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998650    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-var-lib-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998706    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-ovnkube-config\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998743    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-registration-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998783    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-systemd\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998821    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-host\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:19.999011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.998972    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1a089cc1-9715-4809-9479-02fddca15479-ovnkube-config\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:19.999504 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.999048    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed269b7c-e48c-4732-acc4-c0250a95801f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:19.999504 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:19.999112    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b638fc1f-3579-467c-ac54-e4a4bc891f9e-config-volume\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:20.016089 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.016061    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbq66\" (UniqueName: \"kubernetes.io/projected/881291b2-1e3f-491d-8dd8-b1500fc5ab24-kube-api-access-kbq66\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.016357 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.016337    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcsmv\" (UniqueName: \"kubernetes.io/projected/9ae7f604-ca2d-4ddd-8aac-3f3975ef7463-kube-api-access-tcsmv\") pod \"migrator-66df5d496b-s2zkk\" (UID: \"9ae7f604-ca2d-4ddd-8aac-3f3975ef7463\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-s2zkk"
Oct 31 07:29:20.016436 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.016352    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xzg5\" (UniqueName: \"kubernetes.io/projected/ed269b7c-e48c-4732-acc4-c0250a95801f-kube-api-access-4xzg5\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:20.016578 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.016562    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtwtf\" (UniqueName: \"kubernetes.io/projected/1a089cc1-9715-4809-9479-02fddca15479-kube-api-access-xtwtf\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.016613 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.016564    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfsf9\" (UniqueName: \"kubernetes.io/projected/b638fc1f-3579-467c-ac54-e4a4bc891f9e-kube-api-access-vfsf9\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:20.016996 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.016980    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsjgx\" (UniqueName: \"kubernetes.io/projected/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-api-access-zsjgx\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:20.017248 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.017218    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl7mm\" (UniqueName: \"kubernetes.io/projected/e9840048-339a-4d1a-a37c-cdfad40f8883-kube-api-access-sl7mm\") pod \"service-ca-5fffc97bdf-zn7pt\" (UID: \"e9840048-339a-4d1a-a37c-cdfad40f8883\") " pod="openshift-service-ca/service-ca-5fffc97bdf-zn7pt"
Oct 31 07:29:20.018915 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.018893    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-699ww\" (UniqueName: \"kubernetes.io/projected/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-kube-api-access-699ww\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:29:20.024129 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.024113    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4s6h\" (UniqueName: \"kubernetes.io/projected/d95920d0-5a98-46bc-9b64-f820f02d5b1d-kube-api-access-n4s6h\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:29:20.024249 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.024216    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsbrt\" (UniqueName: \"kubernetes.io/projected/05163761-47bd-4b1b-93fc-1ef20691a84f-kube-api-access-vsbrt\") pod \"ingress-canary-vktw2\" (UID: \"05163761-47bd-4b1b-93fc-1ef20691a84f\") " pod="openshift-ingress-canary/ingress-canary-vktw2"
Oct 31 07:29:20.025489 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.025465    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75t9x\" (UniqueName: \"kubernetes.io/projected/0f24a74d-99e3-4bce-a9e1-2eb550655093-kube-api-access-75t9x\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.025566 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.025491    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5624d\" (UniqueName: \"kubernetes.io/projected/cf2415aa-becd-409e-bb51-05d94a380deb-kube-api-access-5624d\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:29:20.025566 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.025552    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-979pp\" (UniqueName: \"kubernetes.io/projected/62191e70-8059-4950-b636-b6c58d33b128-kube-api-access-979pp\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:29:20.025660 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.025618    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vt2p\" (UniqueName: \"kubernetes.io/projected/374fae1c-44be-4729-802f-e190816bf249-kube-api-access-8vt2p\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:20.025703 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.025665    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcn6k\" (UniqueName: \"kubernetes.io/projected/deac7042-7310-4a66-98b3-88fbebb4f2f9-kube-api-access-pcn6k\") pod \"periodic-gathering-k76m7-4srkl\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") " pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:29:20.025815 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.025800    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2978v\" (UniqueName: \"kubernetes.io/projected/72245812-04f1-47b8-a22f-67a61f175b63-kube-api-access-2978v\") pod \"network-check-target-pjddt\" (UID: \"72245812-04f1-47b8-a22f-67a61f175b63\") " pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:29:20.100008 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.099983    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-device-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:20.100008 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100008    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f393f8a-63f4-41d7-aedd-d3df202eff35-crio-socket\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:20.100161 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100023    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-socket-dir-parent\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.100161 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100036    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-etc-selinux\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:20.100161 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100051    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-sys-fs\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:20.100161 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100073    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-multus-certs\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.100161 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100105    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-netns\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.100161 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100124    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-etc-selinux\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:20.100161 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100128    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-hostroot\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.100161 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100125    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-device-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:20.100161 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100137    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-sys-fs\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:20.100161 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100140    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-socket-dir-parent\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.100161 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100161    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f393f8a-63f4-41d7-aedd-d3df202eff35-crio-socket\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100171    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-slash\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100178    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-hostroot\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100193    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-cni-bin\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100213    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-netns\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100179    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-multus-certs\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100218    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-kubernetes\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100260    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-cni-bin\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100271    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysctl-d\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100289    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-kubernetes\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100268    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-slash\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100303    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100330    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-system-cni-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100355    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-node-log\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100384    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-cni-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100408    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62191e70-8059-4950-b636-b6c58d33b128-host\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100430    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-run-netns\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100439    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-node-log\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100450    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-log-socket\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100440    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysctl-d\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100486    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-var-lib-kubelet\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100518    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62191e70-8059-4950-b636-b6c58d33b128-host\") pod \"node-ca-fl4w7\" (UID: \"62191e70-8059-4950-b636-b6c58d33b128\") " pod="openshift-image-registry/node-ca-fl4w7"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100486    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-log-socket\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100524    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-system-cni-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100545    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-cnibin\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100571    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-kubelet\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100586    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-cni-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100598    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-system-cni-dir\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100613    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-kubelet\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100572    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-run-netns\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100573    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-var-lib-kubelet\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100645    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100639    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-system-cni-dir\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100671    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-kubelet\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100703    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-systemd\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.100708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100729    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-socket-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100752    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-run-ovn-kubernetes\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100777    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100799    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-run\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100817    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-socket-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100820    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-sys\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100853    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-cni-multus\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100859    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-sys\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100883    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-systemd\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100932    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-hosts-file\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100939    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f393f8a-63f4-41d7-aedd-d3df202eff35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100903    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-run-ovn-kubernetes\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100752    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-kubelet\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100941    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-cni-multus\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100978    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-run\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100988    4322 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100994    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-cnibin\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.100893    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b1ca5663-7cde-4b55-b971-e2afaf82fb5f-hosts-file\") pod \"node-resolver-gkl8q\" (UID: \"b1ca5663-7cde-4b55-b971-e2afaf82fb5f\") " pod="openshift-dns/node-resolver-gkl8q"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101030    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101042    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-var-lib-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101072    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-registration-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101093    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-var-lib-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101095    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-systemd\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101123    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-host\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101132    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-systemd\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101146    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101166    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-dbus\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101160    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-registration-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101188    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-host\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101185    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-cni-bin\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101214    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-var-lib-cni-bin\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101196    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101252    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101279    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-ovn\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101295    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-modprobe-d\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101313    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101343    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-cnibin\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101324    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-dbus\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101370    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-systemd-units\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101384    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/374fae1c-44be-4729-802f-e190816bf249-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5g95l\" (UID: \"374fae1c-44be-4729-802f-e190816bf249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5g95l"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101393    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-etc-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101399    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-cnibin\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101404    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-modprobe-d\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101418    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-cni-netd\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101421    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-systemd-units\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101444    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysconfig\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101451    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-etc-openvswitch\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101345    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-run-ovn\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101467    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-kubelet-config\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:20.101481    4322 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101495    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-conf-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101498    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysconfig\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101462    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1a089cc1-9715-4809-9479-02fddca15479-host-cni-netd\") pod \"ovnkube-node-vzgl4\" (UID: \"1a089cc1-9715-4809-9479-02fddca15479\") " pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101519    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101520    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/11dc8b0e-286d-42e1-bfe0-d882e7dfe073-kubelet-config\") pod \"global-pull-secret-syncer-z9h8d\" (UID: \"11dc8b0e-286d-42e1-bfe0-d882e7dfe073\") " pod="kube-system/global-pull-secret-syncer-z9h8d"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:20.101552    4322 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls podName:8f393f8a-63f4-41d7-aedd-d3df202eff35 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:20.601533639 +0000 UTC m=+1.865354261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqkwf" (UID: "8f393f8a-63f4-41d7-aedd-d3df202eff35") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101570    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysctl-conf\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101573    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed269b7c-e48c-4732-acc4-c0250a95801f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h4qfb\" (UID: \"ed269b7c-e48c-4732-acc4-c0250a95801f\") " pod="openshift-multus/multus-additional-cni-plugins-h4qfb"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101595    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-lib-modules\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101542    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-multus-conf-dir\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101620    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-k8s-cni-cncf-io\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101643    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-etc-kubernetes\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101667    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf2415aa-becd-409e-bb51-05d94a380deb-host-slash\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101691    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101695    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-etc-sysctl-conf\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101709    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-host-run-k8s-cni-cncf-io\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101706    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/881291b2-1e3f-491d-8dd8-b1500fc5ab24-lib-modules\") pod \"tuned-m7rmk\" (UID: \"881291b2-1e3f-491d-8dd8-b1500fc5ab24\") " pod="openshift-cluster-node-tuning-operator/tuned-m7rmk"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101745    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf2415aa-becd-409e-bb51-05d94a380deb-host-slash\") pod \"iptables-alerter-z22fp\" (UID: \"cf2415aa-becd-409e-bb51-05d94a380deb\") " pod="openshift-network-operator/iptables-alerter-z22fp"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:20.101767    4322 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.101789    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f24a74d-99e3-4bce-a9e1-2eb550655093-etc-kubernetes\") pod \"multus-jghjq\" (UID: \"0f24a74d-99e3-4bce-a9e1-2eb550655093\") " pod="openshift-multus/multus-jghjq"
Oct 31 07:29:20.102076 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:20.101809    4322 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls podName:b638fc1f-3579-467c-ac54-e4a4bc891f9e nodeName:}" failed. No retries permitted until 2026-10-31 07:29:20.601786197 +0000 UTC m=+1.865606823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls") pod "dns-default-w78n9" (UID: "b638fc1f-3579-467c-ac54-e4a4bc891f9e") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:20.104032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.103968    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d95920d0-5a98-46bc-9b64-f820f02d5b1d-metrics-certs\") pod \"network-metrics-daemon-2skf7\" (UID: \"d95920d0-5a98-46bc-9b64-f820f02d5b1d\") " pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:29:20.182353 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.182135    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2skf7"
Oct 31 07:29:20.468648 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.468570    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fl4w7_62191e70-8059-4950-b636-b6c58d33b128/node-ca/0.log"
Oct 31 07:29:20.496760 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.496736    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2skf7"]
Oct 31 07:29:20.501062 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:20.501032    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd95920d0_5a98_46bc_9b64_f820f02d5b1d.slice/crio-507d870026a3716c1166f459445e331eb5e566602d44e03a3ad9429eb0be33c4 WatchSource:0}: Error finding container 507d870026a3716c1166f459445e331eb5e566602d44e03a3ad9429eb0be33c4: Status 404 returned error can't find the container with id 507d870026a3716c1166f459445e331eb5e566602d44e03a3ad9429eb0be33c4
Oct 31 07:29:20.503238 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.503208    4322 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 07:29:20.604542 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.604519    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:20.604635 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.604553    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:20.604672 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:20.604662    4322 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:20.604715 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:20.604705    4322 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls podName:b638fc1f-3579-467c-ac54-e4a4bc891f9e nodeName:}" failed. No retries permitted until 2026-10-31 07:29:21.604694013 +0000 UTC m=+2.868514615 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls") pod "dns-default-w78n9" (UID: "b638fc1f-3579-467c-ac54-e4a4bc891f9e") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:20.604758 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:20.604662    4322 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:20.604804 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:20.604796    4322 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls podName:8f393f8a-63f4-41d7-aedd-d3df202eff35 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:21.604784032 +0000 UTC m=+2.868604634 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqkwf" (UID: "8f393f8a-63f4-41d7-aedd-d3df202eff35") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:20.917650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:20.917610    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2skf7" event={"ID":"d95920d0-5a98-46bc-9b64-f820f02d5b1d","Type":"ContainerStarted","Data":"507d870026a3716c1166f459445e331eb5e566602d44e03a3ad9429eb0be33c4"}
Oct 31 07:29:21.333078 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:21.333046    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:21.467982 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:21.467783    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vktw2_05163761-47bd-4b1b-93fc-1ef20691a84f/serve-healthcheck-canary/0.log"
Oct 31 07:29:21.610556 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:21.610478    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:21.610556 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:21.610518    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:21.610726 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:21.610623    4322 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:21.610726 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:21.610625    4322 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:21.610726 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:21.610672    4322 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls podName:b638fc1f-3579-467c-ac54-e4a4bc891f9e nodeName:}" failed. No retries permitted until 2026-10-31 07:29:23.610659145 +0000 UTC m=+4.874479748 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls") pod "dns-default-w78n9" (UID: "b638fc1f-3579-467c-ac54-e4a4bc891f9e") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:21.610726 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:21.610685    4322 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls podName:8f393f8a-63f4-41d7-aedd-d3df202eff35 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:23.610679673 +0000 UTC m=+4.874500262 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqkwf" (UID: "8f393f8a-63f4-41d7-aedd-d3df202eff35") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:21.868129 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:21.868064    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-s2zkk_9ae7f604-ca2d-4ddd-8aac-3f3975ef7463/migrator/0.log"
Oct 31 07:29:21.923206 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:21.923168    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2skf7" event={"ID":"d95920d0-5a98-46bc-9b64-f820f02d5b1d","Type":"ContainerStarted","Data":"6e7c231ca5aa47bd07559a2f2c944ce410ab793d0f0e9ef21bf4f3985073a0a2"}
Oct 31 07:29:21.923583 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:21.923264    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2skf7" event={"ID":"d95920d0-5a98-46bc-9b64-f820f02d5b1d","Type":"ContainerStarted","Data":"58da3bcd2a4a5b7c501084dae9375182f71b4f1e7252527975fc245f19548d95"}
Oct 31 07:29:23.626069 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:23.626036    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:23.626564 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:23.626078    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:23.626564 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:23.626181    4322 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:23.626564 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:23.626266    4322 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls podName:8f393f8a-63f4-41d7-aedd-d3df202eff35 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:27.62624755 +0000 UTC m=+8.890068159 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqkwf" (UID: "8f393f8a-63f4-41d7-aedd-d3df202eff35") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:23.629777 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:23.629755    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b638fc1f-3579-467c-ac54-e4a4bc891f9e-metrics-tls\") pod \"dns-default-w78n9\" (UID: \"b638fc1f-3579-467c-ac54-e4a4bc891f9e\") " pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:23.786216 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:23.786187    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:24.074426 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:24.074395    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w78n9"]
Oct 31 07:29:24.077870 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:24.077840    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb638fc1f_3579_467c_ac54_e4a4bc891f9e.slice/crio-0088d0b9937380787402ed320901b808cf4fe138546cdeecf46c4df8e5727502 WatchSource:0}: Error finding container 0088d0b9937380787402ed320901b808cf4fe138546cdeecf46c4df8e5727502: Status 404 returned error can't find the container with id 0088d0b9937380787402ed320901b808cf4fe138546cdeecf46c4df8e5727502
Oct 31 07:29:24.936398 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:24.936358    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w78n9" event={"ID":"b638fc1f-3579-467c-ac54-e4a4bc891f9e","Type":"ContainerStarted","Data":"0088d0b9937380787402ed320901b808cf4fe138546cdeecf46c4df8e5727502"}
Oct 31 07:29:25.135435 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:25.135399    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:29:25.941740 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:25.941660    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w78n9" event={"ID":"b638fc1f-3579-467c-ac54-e4a4bc891f9e","Type":"ContainerStarted","Data":"99ba6686fd4b153e21931136f0bb8fc64778fc8a1866577010631ef63a6f35c9"}
Oct 31 07:29:25.941740 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:25.941741    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w78n9" event={"ID":"b638fc1f-3579-467c-ac54-e4a4bc891f9e","Type":"ContainerStarted","Data":"17b3a4a7b94dabb7df7d30ff0c777004096538cec484365edd48378144d926ec"}
Oct 31 07:29:25.942130 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:25.941764    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:26.894893 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:26.894860    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:26.895085 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:26.895038    4322 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 07:29:27.651819 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:27.651778    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:27.653990 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:27.653969    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f393f8a-63f4-41d7-aedd-d3df202eff35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqkwf\" (UID: \"8f393f8a-63f4-41d7-aedd-d3df202eff35\") " pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:27.689952 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:27.689930    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wqkwf"
Oct 31 07:29:27.978610 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:27.978527    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wqkwf"]
Oct 31 07:29:27.982057 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:27.982031    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f393f8a_63f4_41d7_aedd_d3df202eff35.slice/crio-6b220a02849803f61c4084ba34aa482b989b6cc10587b6e3e24a048202267fdf WatchSource:0}: Error finding container 6b220a02849803f61c4084ba34aa482b989b6cc10587b6e3e24a048202267fdf: Status 404 returned error can't find the container with id 6b220a02849803f61c4084ba34aa482b989b6cc10587b6e3e24a048202267fdf
Oct 31 07:29:28.803138 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:28.803110    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vzgl4"
Oct 31 07:29:28.952414 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:28.952326    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wqkwf" event={"ID":"8f393f8a-63f4-41d7-aedd-d3df202eff35","Type":"ContainerStarted","Data":"72d8f76b4d562d75bf14a596f8f706e6991827551cf1d945ccc22d531f0cc5d8"}
Oct 31 07:29:28.952555 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:28.952422    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wqkwf" event={"ID":"8f393f8a-63f4-41d7-aedd-d3df202eff35","Type":"ContainerStarted","Data":"c64704bed8647237cea9c2b731f3c97160698df521fefce6fe3bee1efbd56bef"}
Oct 31 07:29:28.952555 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:28.952446    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wqkwf" event={"ID":"8f393f8a-63f4-41d7-aedd-d3df202eff35","Type":"ContainerStarted","Data":"6b220a02849803f61c4084ba34aa482b989b6cc10587b6e3e24a048202267fdf"}
Oct 31 07:29:29.958739 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:29.958710    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wqkwf" event={"ID":"8f393f8a-63f4-41d7-aedd-d3df202eff35","Type":"ContainerStarted","Data":"fe9cbe2ada542d7c3f9c6019a4096c3742489408429a35605fa7ef1253f619a9"}
Oct 31 07:29:31.709203 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:29:31.709168    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:29:34.533833 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.533798    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-2665c"]
Oct 31 07:29:34.537066 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.537044    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c"
Oct 31 07:29:34.568822 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.568793    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 31 07:29:34.568822 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.568795    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2r2ph\""
Oct 31 07:29:34.569047 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.568987    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 31 07:29:34.569047 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.569014    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 31 07:29:34.569047 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.569017    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 31 07:29:34.569047 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.569036    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 31 07:29:34.581653 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.581629    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-2665c"]
Oct 31 07:29:34.699511 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.699478    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/77485de8-4d57-478d-9544-2263f942e8a6-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-2665c\" (UID: \"77485de8-4d57-478d-9544-2263f942e8a6\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c"
Oct 31 07:29:34.699676 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.699526    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77485de8-4d57-478d-9544-2263f942e8a6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-2665c\" (UID: \"77485de8-4d57-478d-9544-2263f942e8a6\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c"
Oct 31 07:29:34.699676 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.699549    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj9w9\" (UniqueName: \"kubernetes.io/projected/77485de8-4d57-478d-9544-2263f942e8a6-kube-api-access-dj9w9\") pod \"prometheus-operator-9cd89b74c-2665c\" (UID: \"77485de8-4d57-478d-9544-2263f942e8a6\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c"
Oct 31 07:29:34.699676 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.699642    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77485de8-4d57-478d-9544-2263f942e8a6-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-2665c\" (UID: \"77485de8-4d57-478d-9544-2263f942e8a6\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c"
Oct 31 07:29:34.800470 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.800390    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77485de8-4d57-478d-9544-2263f942e8a6-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-2665c\" (UID: \"77485de8-4d57-478d-9544-2263f942e8a6\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c"
Oct 31 07:29:34.800470 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.800450    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/77485de8-4d57-478d-9544-2263f942e8a6-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-2665c\" (UID: \"77485de8-4d57-478d-9544-2263f942e8a6\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c"
Oct 31 07:29:34.800631 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.800497    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77485de8-4d57-478d-9544-2263f942e8a6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-2665c\" (UID: \"77485de8-4d57-478d-9544-2263f942e8a6\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c"
Oct 31 07:29:34.800631 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.800521    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj9w9\" (UniqueName: \"kubernetes.io/projected/77485de8-4d57-478d-9544-2263f942e8a6-kube-api-access-dj9w9\") pod \"prometheus-operator-9cd89b74c-2665c\" (UID: \"77485de8-4d57-478d-9544-2263f942e8a6\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c"
Oct 31 07:29:34.801220 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.801194    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77485de8-4d57-478d-9544-2263f942e8a6-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-2665c\" (UID: \"77485de8-4d57-478d-9544-2263f942e8a6\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c"
Oct 31 07:29:34.802911 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.802889    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77485de8-4d57-478d-9544-2263f942e8a6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-2665c\" (UID: \"77485de8-4d57-478d-9544-2263f942e8a6\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c"
Oct 31 07:29:34.802985 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.802921    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/77485de8-4d57-478d-9544-2263f942e8a6-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-2665c\" (UID: \"77485de8-4d57-478d-9544-2263f942e8a6\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c"
Oct 31 07:29:34.847396 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.847361    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj9w9\" (UniqueName: \"kubernetes.io/projected/77485de8-4d57-478d-9544-2263f942e8a6-kube-api-access-dj9w9\") pod \"prometheus-operator-9cd89b74c-2665c\" (UID: \"77485de8-4d57-478d-9544-2263f942e8a6\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c"
Oct 31 07:29:34.850109 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:34.850084    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c"
Oct 31 07:29:35.140642 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:35.140565    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-2665c"]
Oct 31 07:29:35.143874 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:35.143841    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77485de8_4d57_478d_9544_2263f942e8a6.slice/crio-316978d67e523c97df4276375b2c3e4699b7a6889608da2eb3cc3750286a023a WatchSource:0}: Error finding container 316978d67e523c97df4276375b2c3e4699b7a6889608da2eb3cc3750286a023a: Status 404 returned error can't find the container with id 316978d67e523c97df4276375b2c3e4699b7a6889608da2eb3cc3750286a023a
Oct 31 07:29:35.947345 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:35.947312    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-w78n9"
Oct 31 07:29:35.978083 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:35.978049    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c" event={"ID":"77485de8-4d57-478d-9544-2263f942e8a6","Type":"ContainerStarted","Data":"316978d67e523c97df4276375b2c3e4699b7a6889608da2eb3cc3750286a023a"}
Oct 31 07:29:36.984790 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:36.984756    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c" event={"ID":"77485de8-4d57-478d-9544-2263f942e8a6","Type":"ContainerStarted","Data":"8e4dd3ca13273ba9d14d68d372ae616d30acd7243d6a333a46f2556b11441c38"}
Oct 31 07:29:36.985164 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:36.984808    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c" event={"ID":"77485de8-4d57-478d-9544-2263f942e8a6","Type":"ContainerStarted","Data":"27fa1ef968f4c757c83d928e160b20ad67d3464d906dca763c74dfe282c65214"}
Oct 31 07:29:37.025219 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:37.025164    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-2665c" podStartSLOduration=1.977869062 podStartE2EDuration="3.025150964s" totalImagesPullingTime="1.047281902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:37.024872573 +0000 UTC m=+18.288693183" watchObservedRunningTime="2026-10-31 07:29:37.025150964 +0000 UTC m=+18.288971573"
Oct 31 07:29:39.160716 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.160678    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"]
Oct 31 07:29:39.164164 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.164146    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"
Oct 31 07:29:39.185971 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.185950    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 31 07:29:39.194711 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.194683    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-ftkzs\""
Oct 31 07:29:39.194832 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.194694    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 31 07:29:39.195413 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.195392    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"]
Oct 31 07:29:39.213407 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.213379    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-swsxq"]
Oct 31 07:29:39.216669 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.216645    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.217076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.217055    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"]
Oct 31 07:29:39.220519 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.220501    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.234360 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.234335    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 31 07:29:39.234558 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.234541    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 31 07:29:39.251787 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.251762    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 31 07:29:39.251787 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.251792    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 31 07:29:39.251975 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.251865    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-b5fc6\""
Oct 31 07:29:39.251975 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.251921    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 31 07:29:39.251975 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.251946    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-jkl6s\""
Oct 31 07:29:39.251975 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.251963    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 31 07:29:39.255891 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.255872    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"]
Oct 31 07:29:39.332702 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.332665    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef721f60-2404-4822-b906-44abc26d2b4f-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.332702 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.332699    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wx8m\" (UniqueName: \"kubernetes.io/projected/ef721f60-2404-4822-b906-44abc26d2b4f-kube-api-access-8wx8m\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.332953 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.332719    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab956a0c-6d92-4fb9-9b24-98afb9ecf44e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-2vtpg\" (UID: \"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"
Oct 31 07:29:39.332953 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.332780    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b806130-90d2-4c64-8c8b-f81a64560850-metrics-client-ca\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.332953 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.332890    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0b806130-90d2-4c64-8c8b-f81a64560850-node-exporter-accelerators-collector-config\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.332953 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.332930    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ef721f60-2404-4822-b906-44abc26d2b4f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.333108 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.332974    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b806130-90d2-4c64-8c8b-f81a64560850-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.333108 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.333001    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0b806130-90d2-4c64-8c8b-f81a64560850-node-exporter-wtmp\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.333108 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.333043    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab956a0c-6d92-4fb9-9b24-98afb9ecf44e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-2vtpg\" (UID: \"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"
Oct 31 07:29:39.333108 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.333063    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0b806130-90d2-4c64-8c8b-f81a64560850-node-exporter-textfile\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.333108 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.333080    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hwlp\" (UniqueName: \"kubernetes.io/projected/0b806130-90d2-4c64-8c8b-f81a64560850-kube-api-access-7hwlp\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.333108 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.333098    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ef721f60-2404-4822-b906-44abc26d2b4f-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.333389 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.333186    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0b806130-90d2-4c64-8c8b-f81a64560850-node-exporter-tls\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.333389 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.333217    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef721f60-2404-4822-b906-44abc26d2b4f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.333389 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.333292    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0b806130-90d2-4c64-8c8b-f81a64560850-root\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.333389 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.333320    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9wj6\" (UniqueName: \"kubernetes.io/projected/ab956a0c-6d92-4fb9-9b24-98afb9ecf44e-kube-api-access-r9wj6\") pod \"openshift-state-metrics-7bd7489957-2vtpg\" (UID: \"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"
Oct 31 07:29:39.333389 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.333340    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef721f60-2404-4822-b906-44abc26d2b4f-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.333389 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.333363    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab956a0c-6d92-4fb9-9b24-98afb9ecf44e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-2vtpg\" (UID: \"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"
Oct 31 07:29:39.333576 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.333401    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b806130-90d2-4c64-8c8b-f81a64560850-sys\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.363320 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.363297    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6ll6x"
Oct 31 07:29:39.434004 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.433937    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef721f60-2404-4822-b906-44abc26d2b4f-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.434004 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.433966    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wx8m\" (UniqueName: \"kubernetes.io/projected/ef721f60-2404-4822-b906-44abc26d2b4f-kube-api-access-8wx8m\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.434004 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.433986    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab956a0c-6d92-4fb9-9b24-98afb9ecf44e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-2vtpg\" (UID: \"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"
Oct 31 07:29:39.434004 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434003    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b806130-90d2-4c64-8c8b-f81a64560850-metrics-client-ca\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.434319 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434023    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0b806130-90d2-4c64-8c8b-f81a64560850-node-exporter-accelerators-collector-config\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.434319 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434043    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ef721f60-2404-4822-b906-44abc26d2b4f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.434319 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434068    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b806130-90d2-4c64-8c8b-f81a64560850-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.434319 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434094    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0b806130-90d2-4c64-8c8b-f81a64560850-node-exporter-wtmp\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.434319 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434119    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab956a0c-6d92-4fb9-9b24-98afb9ecf44e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-2vtpg\" (UID: \"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"
Oct 31 07:29:39.434319 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434148    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0b806130-90d2-4c64-8c8b-f81a64560850-node-exporter-textfile\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.434319 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434171    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hwlp\" (UniqueName: \"kubernetes.io/projected/0b806130-90d2-4c64-8c8b-f81a64560850-kube-api-access-7hwlp\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.434319 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434198    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ef721f60-2404-4822-b906-44abc26d2b4f-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.434319 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434248    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0b806130-90d2-4c64-8c8b-f81a64560850-node-exporter-tls\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.434319 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434275    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef721f60-2404-4822-b906-44abc26d2b4f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.434319 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434301    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0b806130-90d2-4c64-8c8b-f81a64560850-root\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.434819 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434341    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9wj6\" (UniqueName: \"kubernetes.io/projected/ab956a0c-6d92-4fb9-9b24-98afb9ecf44e-kube-api-access-r9wj6\") pod \"openshift-state-metrics-7bd7489957-2vtpg\" (UID: \"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"
Oct 31 07:29:39.434819 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434369    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef721f60-2404-4822-b906-44abc26d2b4f-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.434819 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434394    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab956a0c-6d92-4fb9-9b24-98afb9ecf44e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-2vtpg\" (UID: \"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"
Oct 31 07:29:39.434819 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434422    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b806130-90d2-4c64-8c8b-f81a64560850-sys\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.434819 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434504    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b806130-90d2-4c64-8c8b-f81a64560850-sys\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.434819 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434555    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0b806130-90d2-4c64-8c8b-f81a64560850-root\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.434819 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434770    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b806130-90d2-4c64-8c8b-f81a64560850-metrics-client-ca\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.435166 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434917    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab956a0c-6d92-4fb9-9b24-98afb9ecf44e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-2vtpg\" (UID: \"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"
Oct 31 07:29:39.435166 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434925    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ef721f60-2404-4822-b906-44abc26d2b4f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.435166 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.434947    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0b806130-90d2-4c64-8c8b-f81a64560850-node-exporter-accelerators-collector-config\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.435166 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.435068    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0b806130-90d2-4c64-8c8b-f81a64560850-node-exporter-wtmp\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.435166 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.435073    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ef721f60-2404-4822-b906-44abc26d2b4f-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.435443 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.435414    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0b806130-90d2-4c64-8c8b-f81a64560850-node-exporter-textfile\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.435734 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.435676    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef721f60-2404-4822-b906-44abc26d2b4f-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.436937 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.436883    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef721f60-2404-4822-b906-44abc26d2b4f-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.437220 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.437182    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab956a0c-6d92-4fb9-9b24-98afb9ecf44e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-2vtpg\" (UID: \"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"
Oct 31 07:29:39.437311 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.437215    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef721f60-2404-4822-b906-44abc26d2b4f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.437543 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.437527    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b806130-90d2-4c64-8c8b-f81a64560850-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.437627 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.437610    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0b806130-90d2-4c64-8c8b-f81a64560850-node-exporter-tls\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.437723 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.437706    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab956a0c-6d92-4fb9-9b24-98afb9ecf44e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-2vtpg\" (UID: \"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"
Oct 31 07:29:39.456555 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.456523    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wx8m\" (UniqueName: \"kubernetes.io/projected/ef721f60-2404-4822-b906-44abc26d2b4f-kube-api-access-8wx8m\") pod \"kube-state-metrics-5dc8d7d89b-8k49p\" (UID: \"ef721f60-2404-4822-b906-44abc26d2b4f\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.457178 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.457152    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hwlp\" (UniqueName: \"kubernetes.io/projected/0b806130-90d2-4c64-8c8b-f81a64560850-kube-api-access-7hwlp\") pod \"node-exporter-swsxq\" (UID: \"0b806130-90d2-4c64-8c8b-f81a64560850\") " pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.457282 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.457163    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9wj6\" (UniqueName: \"kubernetes.io/projected/ab956a0c-6d92-4fb9-9b24-98afb9ecf44e-kube-api-access-r9wj6\") pod \"openshift-state-metrics-7bd7489957-2vtpg\" (UID: \"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"
Oct 31 07:29:39.479005 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.478969    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"
Oct 31 07:29:39.533906 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.533879    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-swsxq"
Oct 31 07:29:39.540598 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.540570    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"
Oct 31 07:29:39.550650 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:39.550625    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b806130_90d2_4c64_8c8b_f81a64560850.slice/crio-de71230e00baef5440b741c24bae67231a8a692150a2598720a2c4e0aebcc553 WatchSource:0}: Error finding container de71230e00baef5440b741c24bae67231a8a692150a2598720a2c4e0aebcc553: Status 404 returned error can't find the container with id de71230e00baef5440b741c24bae67231a8a692150a2598720a2c4e0aebcc553
Oct 31 07:29:39.779197 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.779151    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg"]
Oct 31 07:29:39.782637 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:39.782610    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab956a0c_6d92_4fb9_9b24_98afb9ecf44e.slice/crio-7e5b497e0528d53235675aaa8b116f086377fbbdeeff8491248f6096fdf8207b WatchSource:0}: Error finding container 7e5b497e0528d53235675aaa8b116f086377fbbdeeff8491248f6096fdf8207b: Status 404 returned error can't find the container with id 7e5b497e0528d53235675aaa8b116f086377fbbdeeff8491248f6096fdf8207b
Oct 31 07:29:39.855647 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.855599    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p"]
Oct 31 07:29:39.859967 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:39.859935    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef721f60_2404_4822_b906_44abc26d2b4f.slice/crio-e49be9158257396b70ca88ab48cbb9e74ddb3cc3812b0287ce5500cff8157494 WatchSource:0}: Error finding container e49be9158257396b70ca88ab48cbb9e74ddb3cc3812b0287ce5500cff8157494: Status 404 returned error can't find the container with id e49be9158257396b70ca88ab48cbb9e74ddb3cc3812b0287ce5500cff8157494
Oct 31 07:29:39.995363 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.995316    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p" event={"ID":"ef721f60-2404-4822-b906-44abc26d2b4f","Type":"ContainerStarted","Data":"e49be9158257396b70ca88ab48cbb9e74ddb3cc3812b0287ce5500cff8157494"}
Oct 31 07:29:39.996681 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.996650    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-swsxq" event={"ID":"0b806130-90d2-4c64-8c8b-f81a64560850","Type":"ContainerStarted","Data":"de71230e00baef5440b741c24bae67231a8a692150a2598720a2c4e0aebcc553"}
Oct 31 07:29:39.998789 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.998766    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg" event={"ID":"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e","Type":"ContainerStarted","Data":"cba2a1b27e2d49605b4640293f03230b6115e5d3ecf2de31ad23355edbb220c6"}
Oct 31 07:29:39.998906 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.998817    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg" event={"ID":"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e","Type":"ContainerStarted","Data":"997e99dc347fb60f3df48c3d7469b35f5c5cfa855db26980e71a3e29c5e1b931"}
Oct 31 07:29:39.998906 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:39.998842    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg" event={"ID":"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e","Type":"ContainerStarted","Data":"7e5b497e0528d53235675aaa8b116f086377fbbdeeff8491248f6096fdf8207b"}
Oct 31 07:29:41.003329 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.003290    4322 generic.go:356] "Generic (PLEG): container finished" podID="0b806130-90d2-4c64-8c8b-f81a64560850" containerID="852c9e7cd3c6e3ba325295fee3df8605a3a2f9669a7a50bd2649c6bbe54c5606" exitCode=0
Oct 31 07:29:41.003757 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.003364    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-swsxq" event={"ID":"0b806130-90d2-4c64-8c8b-f81a64560850","Type":"ContainerDied","Data":"852c9e7cd3c6e3ba325295fee3df8605a3a2f9669a7a50bd2649c6bbe54c5606"}
Oct 31 07:29:41.005582 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.005553    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg" event={"ID":"ab956a0c-6d92-4fb9-9b24-98afb9ecf44e","Type":"ContainerStarted","Data":"6a8a0d4ed817cfcea91a30f95f70a120c5c7ae73a65330d0ba5dbb77d12a1c17"}
Oct 31 07:29:41.088571 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.088516    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2vtpg" podStartSLOduration=1.163446219 podStartE2EDuration="2.088497741s" totalImagesPullingTime="925.051522ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:41.08744067 +0000 UTC m=+22.351261281" watchObservedRunningTime="2026-10-31 07:29:41.088497741 +0000 UTC m=+22.352318352"
Oct 31 07:29:41.455035 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.454956    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-564c757d94-dl45p"]
Oct 31 07:29:41.459179 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.459151    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.480125 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.480095    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 31 07:29:41.480125 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.480099    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 31 07:29:41.480352 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.480319    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 31 07:29:41.480352 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.480329    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 31 07:29:41.480430 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.480363    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 31 07:29:41.480430 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.480391    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vx842\""
Oct 31 07:29:41.480430 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.480394    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fd0r36dpaea5d\""
Oct 31 07:29:41.486000 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.485979    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-564c757d94-dl45p"]
Oct 31 07:29:41.548239 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.548199    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q5kt\" (UniqueName: \"kubernetes.io/projected/36f67793-b79a-4fa6-bb3f-676e694f263e-kube-api-access-8q5kt\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.548439 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.548271    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-thanos-querier-tls\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.548439 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.548325    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-grpc-tls\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.548439 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.548415    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.548575 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.548456    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.548575 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.548482    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.548575 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.548523    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.548660 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.548608    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36f67793-b79a-4fa6-bb3f-676e694f263e-metrics-client-ca\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.548660 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.548633    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36f67793-b79a-4fa6-bb3f-676e694f263e-tmp\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.649590 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.649547    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q5kt\" (UniqueName: \"kubernetes.io/projected/36f67793-b79a-4fa6-bb3f-676e694f263e-kube-api-access-8q5kt\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.649777 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.649607    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-thanos-querier-tls\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.649777 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.649642    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-grpc-tls\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.649777 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.649674    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.649777 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.649703    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.649777 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.649730    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.649777 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.649767    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.650081 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.649838    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36f67793-b79a-4fa6-bb3f-676e694f263e-metrics-client-ca\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.650081 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.649867    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36f67793-b79a-4fa6-bb3f-676e694f263e-tmp\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.650513 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.650488    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36f67793-b79a-4fa6-bb3f-676e694f263e-tmp\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.650725 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.650700    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36f67793-b79a-4fa6-bb3f-676e694f263e-metrics-client-ca\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.652713 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.652662    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-grpc-tls\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.653062 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.653037    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-thanos-querier-tls\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.653062 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.653047    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.653273 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.653177    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.653273 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.653200    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.653461 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.653440    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/36f67793-b79a-4fa6-bb3f-676e694f263e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.672280 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.672252    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q5kt\" (UniqueName: \"kubernetes.io/projected/36f67793-b79a-4fa6-bb3f-676e694f263e-kube-api-access-8q5kt\") pod \"thanos-querier-564c757d94-dl45p\" (UID: \"36f67793-b79a-4fa6-bb3f-676e694f263e\") " pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:41.782327 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:41.782295    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:42.015643 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:42.015602    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p" event={"ID":"ef721f60-2404-4822-b906-44abc26d2b4f","Type":"ContainerStarted","Data":"051c86a3e8b8b94330ce0876efaccc160929e91292bce7db66a7021b9f8d715a"}
Oct 31 07:29:42.016064 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:42.015697    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p" event={"ID":"ef721f60-2404-4822-b906-44abc26d2b4f","Type":"ContainerStarted","Data":"f0a8b11e06517e6792b4cfec4379ba0c96f25416349d419bd604005dc120d5d6"}
Oct 31 07:29:42.016064 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:42.015723    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p" event={"ID":"ef721f60-2404-4822-b906-44abc26d2b4f","Type":"ContainerStarted","Data":"5ccebc57be163c0fa03708ada2caf965083d3ca69074be0b07571e14a91edf56"}
Oct 31 07:29:42.017725 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:42.017698    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-swsxq" event={"ID":"0b806130-90d2-4c64-8c8b-f81a64560850","Type":"ContainerStarted","Data":"3fb5e42cb23ae26b00a13cc0817f73ec993c0eeb48e2a3c80cae534ecf758de2"}
Oct 31 07:29:42.017830 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:42.017743    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-swsxq" event={"ID":"0b806130-90d2-4c64-8c8b-f81a64560850","Type":"ContainerStarted","Data":"4e9d5203431ad83db600fd5722d435ec13fd4deb8bc1fe1d187f4ac5fa30af96"}
Oct 31 07:29:42.057554 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:42.057468    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8k49p" podStartSLOduration=1.8641807190000002 podStartE2EDuration="3.057456183s" totalImagesPullingTime="1.193275464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:42.057189724 +0000 UTC m=+23.321010327" watchObservedRunningTime="2026-10-31 07:29:42.057456183 +0000 UTC m=+23.321276788"
Oct 31 07:29:42.101177 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:42.101139    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-564c757d94-dl45p"]
Oct 31 07:29:42.103433 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:42.103392    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-swsxq" podStartSLOduration=2.075207366 podStartE2EDuration="3.103381062s" totalImagesPullingTime="952.823031ms" totalInitContainerRuntime="75.350665ms" isStatefulPod=true podCreationTimestamp="2026-10-31 07:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:42.103374263 +0000 UTC m=+23.367194874" watchObservedRunningTime="2026-10-31 07:29:42.103381062 +0000 UTC m=+23.367201670"
Oct 31 07:29:42.104907 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:42.104889    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f67793_b79a_4fa6_bb3f_676e694f263e.slice/crio-3978910824d0f2f9d30bcdeb793f0bb4a2b6f87bf23f85445e105ef443b43741 WatchSource:0}: Error finding container 3978910824d0f2f9d30bcdeb793f0bb4a2b6f87bf23f85445e105ef443b43741: Status 404 returned error can't find the container with id 3978910824d0f2f9d30bcdeb793f0bb4a2b6f87bf23f85445e105ef443b43741
Oct 31 07:29:43.022432 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.022390    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564c757d94-dl45p" event={"ID":"36f67793-b79a-4fa6-bb3f-676e694f263e","Type":"ContainerStarted","Data":"3978910824d0f2f9d30bcdeb793f0bb4a2b6f87bf23f85445e105ef443b43741"}
Oct 31 07:29:43.669843 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.669809    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5b647558c9-hg4hd"]
Oct 31 07:29:43.673008 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.672986    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.688575 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.688553    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 31 07:29:43.700387 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.700362    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 31 07:29:43.700546 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.700403    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 31 07:29:43.700546 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.700442    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-sjrww\""
Oct 31 07:29:43.700691 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.700614    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 31 07:29:43.700691 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.700639    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fhj7dqe6n56dl\""
Oct 31 07:29:43.701585 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.701523    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b647558c9-hg4hd"]
Oct 31 07:29:43.767411 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.767378    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/64524a76-3cb4-469e-a086-e7c6fad149b1-secret-metrics-server-tls\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.767577 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.767424    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/64524a76-3cb4-469e-a086-e7c6fad149b1-client-ca-bundle\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.767577 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.767454    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/64524a76-3cb4-469e-a086-e7c6fad149b1-secret-metrics-server-client-certs\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.767702 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.767600    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/64524a76-3cb4-469e-a086-e7c6fad149b1-metrics-server-audit-profiles\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.767702 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.767666    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64524a76-3cb4-469e-a086-e7c6fad149b1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.767702 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.767698    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/64524a76-3cb4-469e-a086-e7c6fad149b1-audit-log\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.767836 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.767716    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khjgc\" (UniqueName: \"kubernetes.io/projected/64524a76-3cb4-469e-a086-e7c6fad149b1-kube-api-access-khjgc\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.868127 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.868099    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64524a76-3cb4-469e-a086-e7c6fad149b1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.868248 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.868150    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/64524a76-3cb4-469e-a086-e7c6fad149b1-audit-log\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.868248 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.868175    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khjgc\" (UniqueName: \"kubernetes.io/projected/64524a76-3cb4-469e-a086-e7c6fad149b1-kube-api-access-khjgc\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.868248 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.868216    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/64524a76-3cb4-469e-a086-e7c6fad149b1-secret-metrics-server-tls\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.868467 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.868260    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/64524a76-3cb4-469e-a086-e7c6fad149b1-client-ca-bundle\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.868467 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.868280    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/64524a76-3cb4-469e-a086-e7c6fad149b1-secret-metrics-server-client-certs\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.868467 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.868302    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/64524a76-3cb4-469e-a086-e7c6fad149b1-metrics-server-audit-profiles\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.868622 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.868581    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/64524a76-3cb4-469e-a086-e7c6fad149b1-audit-log\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.868846 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.868825    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64524a76-3cb4-469e-a086-e7c6fad149b1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.869350 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.869324    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/64524a76-3cb4-469e-a086-e7c6fad149b1-metrics-server-audit-profiles\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.871164 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.871141    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/64524a76-3cb4-469e-a086-e7c6fad149b1-client-ca-bundle\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.871351 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.871331    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/64524a76-3cb4-469e-a086-e7c6fad149b1-secret-metrics-server-client-certs\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.871620 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.871599    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/64524a76-3cb4-469e-a086-e7c6fad149b1-secret-metrics-server-tls\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.893401 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.893379    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khjgc\" (UniqueName: \"kubernetes.io/projected/64524a76-3cb4-469e-a086-e7c6fad149b1-kube-api-access-khjgc\") pod \"metrics-server-5b647558c9-hg4hd\" (UID: \"64524a76-3cb4-469e-a086-e7c6fad149b1\") " pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:43.988473 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:43.988446    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:29:44.031260 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.029282    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564c757d94-dl45p" event={"ID":"36f67793-b79a-4fa6-bb3f-676e694f263e","Type":"ContainerStarted","Data":"efd37b0e712d70f5076bb15dc8d31b8ae60e86ea94670f2e52aee8b702a7a171"}
Oct 31 07:29:44.031260 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.029351    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564c757d94-dl45p" event={"ID":"36f67793-b79a-4fa6-bb3f-676e694f263e","Type":"ContainerStarted","Data":"0d04e1f7cdaf80f1979e8e75482fb0bead60f8e48c10d27bd7fb431c371f4223"}
Oct 31 07:29:44.031260 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.029377    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564c757d94-dl45p" event={"ID":"36f67793-b79a-4fa6-bb3f-676e694f263e","Type":"ContainerStarted","Data":"a0ecc3c7754e55a68d8a2d8ff1d420fc6488f50abdf9a5dbe1ead9a7d89f11a8"}
Oct 31 07:29:44.295892 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.295864    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b647558c9-hg4hd"]
Oct 31 07:29:44.298576 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:44.298539    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64524a76_3cb4_469e_a086_e7c6fad149b1.slice/crio-015b09ff4624685f23cdf52056cfbddc6abd0d02ff1ecbd020f531dd8f35353f WatchSource:0}: Error finding container 015b09ff4624685f23cdf52056cfbddc6abd0d02ff1ecbd020f531dd8f35353f: Status 404 returned error can't find the container with id 015b09ff4624685f23cdf52056cfbddc6abd0d02ff1ecbd020f531dd8f35353f
Oct 31 07:29:44.535560 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.535524    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-77dbb7489d-x64wh"]
Oct 31 07:29:44.539481 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.539457    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.557131 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.557104    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 31 07:29:44.557285 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.557199    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 31 07:29:44.557285 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.557244    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 31 07:29:44.557285 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.557277    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-7k5v9\""
Oct 31 07:29:44.557420 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.557267    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 31 07:29:44.557420 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.557305    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 31 07:29:44.585383 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.583702    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 31 07:29:44.585826 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.585556    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77dbb7489d-x64wh"]
Oct 31 07:29:44.678137 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.678113    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-secret-telemeter-client\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.678266 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.678143    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-telemeter-client-tls\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.678266 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.678166    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.678266 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.678247    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.678383 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.678318    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-metrics-client-ca\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.678383 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.678342    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-serving-certs-ca-bundle\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.678383 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.678368    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-federate-client-tls\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.678485 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.678396    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs4q6\" (UniqueName: \"kubernetes.io/projected/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-kube-api-access-xs4q6\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.779208 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.779170    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-metrics-client-ca\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.779392 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.779237    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-serving-certs-ca-bundle\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.779392 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.779281    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-federate-client-tls\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.779392 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.779316    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs4q6\" (UniqueName: \"kubernetes.io/projected/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-kube-api-access-xs4q6\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.779392 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.779369    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-secret-telemeter-client\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.779392 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.779391    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-telemeter-client-tls\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.779610 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.779412    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.779610 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.779448    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.780049 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.780015    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-metrics-client-ca\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.780359 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.780337    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-serving-certs-ca-bundle\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.780492 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.780417    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.781910 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.781880    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-secret-telemeter-client\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.782256 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.782218    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.782784 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.782767    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-telemeter-client-tls\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.782870 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.782852    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-federate-client-tls\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.802921 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.802861    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs4q6\" (UniqueName: \"kubernetes.io/projected/e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5-kube-api-access-xs4q6\") pod \"telemeter-client-77dbb7489d-x64wh\" (UID: \"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5\") " pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:44.856751 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:44.856716    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh"
Oct 31 07:29:45.036187 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:45.036153    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564c757d94-dl45p" event={"ID":"36f67793-b79a-4fa6-bb3f-676e694f263e","Type":"ContainerStarted","Data":"ccfef7d159457c9ad6ef0cfa95a436052af10a4409cc61d493c85c6d76465266"}
Oct 31 07:29:45.036717 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:45.036220    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564c757d94-dl45p" event={"ID":"36f67793-b79a-4fa6-bb3f-676e694f263e","Type":"ContainerStarted","Data":"0a43869722bf84e01027d5f3e6a5610979a7258d96bdc12db3907dd5b49af093"}
Oct 31 07:29:45.036717 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:45.036282    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-564c757d94-dl45p" event={"ID":"36f67793-b79a-4fa6-bb3f-676e694f263e","Type":"ContainerStarted","Data":"aee3b64a1f3b36be95961ac2af8156750af28f4c6c9a79c7c50e4757560d79f4"}
Oct 31 07:29:45.036717 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:45.036328    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:45.037518 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:45.037490    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd" event={"ID":"64524a76-3cb4-469e-a086-e7c6fad149b1","Type":"ContainerStarted","Data":"015b09ff4624685f23cdf52056cfbddc6abd0d02ff1ecbd020f531dd8f35353f"}
Oct 31 07:29:45.085043 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:45.084956    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-564c757d94-dl45p" podStartSLOduration=1.8098682780000002 podStartE2EDuration="4.084942437s" totalImagesPullingTime="2.275074159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:45.083853528 +0000 UTC m=+26.347674138" watchObservedRunningTime="2026-10-31 07:29:45.084942437 +0000 UTC m=+26.348763047"
Oct 31 07:29:45.173473 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:45.173396    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77dbb7489d-x64wh"]
Oct 31 07:29:45.177523 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:45.177488    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode642f0a5_36c7_4d08_8f2e_ba00e9dc1cb5.slice/crio-fa79ed4004ddb25c73bc17addf3ca3d44bd4174a438aeb884ee4302641c483b4 WatchSource:0}: Error finding container fa79ed4004ddb25c73bc17addf3ca3d44bd4174a438aeb884ee4302641c483b4: Status 404 returned error can't find the container with id fa79ed4004ddb25c73bc17addf3ca3d44bd4174a438aeb884ee4302641c483b4
Oct 31 07:29:46.043746 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:46.043709    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd" event={"ID":"64524a76-3cb4-469e-a086-e7c6fad149b1","Type":"ContainerStarted","Data":"bd505f4d9f716f6eb7513202043f1cb0164e5b3c73cb4b198b67eae0ccc43b62"}
Oct 31 07:29:46.045038 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:46.045009    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh" event={"ID":"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5","Type":"ContainerStarted","Data":"fa79ed4004ddb25c73bc17addf3ca3d44bd4174a438aeb884ee4302641c483b4"}
Oct 31 07:29:46.085795 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:46.085749    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd" podStartSLOduration=1.908575489 podStartE2EDuration="3.085732708s" totalImagesPullingTime="1.177157219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:46.084721622 +0000 UTC m=+27.348542241" watchObservedRunningTime="2026-10-31 07:29:46.085732708 +0000 UTC m=+27.349553320"
Oct 31 07:29:47.050292 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.050256    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh" event={"ID":"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5","Type":"ContainerStarted","Data":"4baf9f3689c568f0692232a65a8930d4fad81e1b98454c332480d887f23cc9e3"}
Oct 31 07:29:47.546565 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.546535    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-69496b9db8-5vsmv"]
Oct 31 07:29:47.552418 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.552391    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.582669 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.582642    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 31 07:29:47.582669 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.582643    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 31 07:29:47.582870 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.582742    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 31 07:29:47.582870 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.582777    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 31 07:29:47.594764 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.594733    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 31 07:29:47.594764 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.594735    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 31 07:29:47.594764 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.594743    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 31 07:29:47.595022 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.594979    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pnf7c\""
Oct 31 07:29:47.595805 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.595788    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69496b9db8-5vsmv"]
Oct 31 07:29:47.605336 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.605317    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-service-ca\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.605430 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.605346    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed04f535-aa10-41f3-8d08-f5a9759b8bea-tmp\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.605430 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.605375    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-oauth-config\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.605430 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.605396    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-config\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.605591 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.605447    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxrtq\" (UniqueName: \"kubernetes.io/projected/ed04f535-aa10-41f3-8d08-f5a9759b8bea-kube-api-access-sxrtq\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.605591 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.605521    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-oauth-serving-cert\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.605591 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.605545    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-trusted-ca-bundle\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.605591 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.605564    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-serving-cert\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.607315 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.607298    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 31 07:29:47.706150 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.706118    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-oauth-config\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.706272 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.706159    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-config\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.706272 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.706190    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxrtq\" (UniqueName: \"kubernetes.io/projected/ed04f535-aa10-41f3-8d08-f5a9759b8bea-kube-api-access-sxrtq\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.706272 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.706259    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-oauth-serving-cert\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.706438 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.706302    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-trusted-ca-bundle\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.706438 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.706340    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-serving-cert\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.706438 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.706381    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-service-ca\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.706438 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.706408    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed04f535-aa10-41f3-8d08-f5a9759b8bea-tmp\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.706996 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.706968    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-oauth-serving-cert\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.707120 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.707040    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-config\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.707181 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.707161    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-service-ca\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.707313 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.707291    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-trusted-ca-bundle\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.707925 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.707906    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed04f535-aa10-41f3-8d08-f5a9759b8bea-tmp\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.709025 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.709000    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-oauth-config\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.709291 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.709275    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-serving-cert\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.728620 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.728599    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxrtq\" (UniqueName: \"kubernetes.io/projected/ed04f535-aa10-41f3-8d08-f5a9759b8bea-kube-api-access-sxrtq\") pod \"console-69496b9db8-5vsmv\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") " pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:47.864134 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:47.864067    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:29:48.055788 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:48.055752    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh" event={"ID":"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5","Type":"ContainerStarted","Data":"180055d0e21442d67f52719384f8d585d7ac648a91ebdc8bda862aef6ba519c4"}
Oct 31 07:29:48.056216 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:48.055812    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh" event={"ID":"e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5","Type":"ContainerStarted","Data":"9fc353cbeab3c09fbe42b187d3f9e812d20e43daa8d7c125b87778bc274d91d6"}
Oct 31 07:29:48.101881 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:48.101826    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-77dbb7489d-x64wh" podStartSLOduration=1.730755534 podStartE2EDuration="4.101809991s" totalImagesPullingTime="2.371054457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:48.101074378 +0000 UTC m=+29.364894992" watchObservedRunningTime="2026-10-31 07:29:48.101809991 +0000 UTC m=+29.365630599"
Oct 31 07:29:48.170828 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:48.170755    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69496b9db8-5vsmv"]
Oct 31 07:29:48.174381 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:29:48.174355    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded04f535_aa10_41f3_8d08_f5a9759b8bea.slice/crio-546d631c5686bdd5e21c4eb60dcb9545b06a3122fccab02db174e507f76f388c WatchSource:0}: Error finding container 546d631c5686bdd5e21c4eb60dcb9545b06a3122fccab02db174e507f76f388c: Status 404 returned error can't find the container with id 546d631c5686bdd5e21c4eb60dcb9545b06a3122fccab02db174e507f76f388c
Oct 31 07:29:49.064132 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:49.064090    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69496b9db8-5vsmv" event={"ID":"ed04f535-aa10-41f3-8d08-f5a9759b8bea","Type":"ContainerStarted","Data":"546d631c5686bdd5e21c4eb60dcb9545b06a3122fccab02db174e507f76f388c"}
Oct 31 07:29:51.052316 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:51.052289    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-564c757d94-dl45p"
Oct 31 07:29:52.076247 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:52.076200    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69496b9db8-5vsmv" event={"ID":"ed04f535-aa10-41f3-8d08-f5a9759b8bea","Type":"ContainerStarted","Data":"d60a785ccbc61af7dbaddbb16914249dbdbf2d75fc8b8e0f839e509a64d4c6ef"}
Oct 31 07:29:52.129132 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:52.129078    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-69496b9db8-5vsmv" podStartSLOduration=1.949731168 podStartE2EDuration="5.129064401s" totalImagesPullingTime="3.179333233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:52.122675276 +0000 UTC m=+33.386495886" watchObservedRunningTime="2026-10-31 07:29:52.129064401 +0000 UTC m=+33.392885012"
Oct 31 07:29:54.556137 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:54.556102    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69496b9db8-5vsmv"]
Oct 31 07:29:55.137776 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:55.137743    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pjddt"
Oct 31 07:29:57.864410 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:29:57.864381    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:30:01.710088 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:30:01.709962    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:30:03.989278 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:03.989241    4322 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:30:03.989851 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:03.989835    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:30:19.575758 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.575702    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-69496b9db8-5vsmv" podUID="ed04f535-aa10-41f3-8d08-f5a9759b8bea" containerName="console" containerID="cri-o://d60a785ccbc61af7dbaddbb16914249dbdbf2d75fc8b8e0f839e509a64d4c6ef" gracePeriod=15
Oct 31 07:30:19.831078 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.831027    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:30:19.988377 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.988344    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-oauth-config\") pod \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") "
Oct 31 07:30:19.988377 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.988381    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-trusted-ca-bundle\") pod \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") "
Oct 31 07:30:19.988611 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.988403    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-serving-cert\") pod \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") "
Oct 31 07:30:19.988674 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.988606    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed04f535-aa10-41f3-8d08-f5a9759b8bea-tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed04f535-aa10-41f3-8d08-f5a9759b8bea-tmp\") pod \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") "
Oct 31 07:30:19.988730 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.988676    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed04f535-aa10-41f3-8d08-f5a9759b8bea-kube-api-access-sxrtq\" (UniqueName: \"kubernetes.io/projected/ed04f535-aa10-41f3-8d08-f5a9759b8bea-kube-api-access-sxrtq\") pod \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") "
Oct 31 07:30:19.988730 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.988714    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-oauth-serving-cert\") pod \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") "
Oct 31 07:30:19.988831 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.988792    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-config\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-config\") pod \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") "
Oct 31 07:30:19.988889 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.988826    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-service-ca\") pod \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\" (UID: \"ed04f535-aa10-41f3-8d08-f5a9759b8bea\") "
Oct 31 07:30:19.988937 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.988910    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed04f535-aa10-41f3-8d08-f5a9759b8bea-tmp" pod "ed04f535-aa10-41f3-8d08-f5a9759b8bea" (UID: "ed04f535-aa10-41f3-8d08-f5a9759b8bea"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:30:19.989070 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.988950    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-trusted-ca-bundle" pod "ed04f535-aa10-41f3-8d08-f5a9759b8bea" (UID: "ed04f535-aa10-41f3-8d08-f5a9759b8bea"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:30:19.989149 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.989127    4322 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed04f535-aa10-41f3-8d08-f5a9759b8bea-tmp\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:30:19.989204 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.989145    4322 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-trusted-ca-bundle\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:30:19.989283 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.989261    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-service-ca" pod "ed04f535-aa10-41f3-8d08-f5a9759b8bea" (UID: "ed04f535-aa10-41f3-8d08-f5a9759b8bea"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:30:19.989333 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.989307    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-config" pod "ed04f535-aa10-41f3-8d08-f5a9759b8bea" (UID: "ed04f535-aa10-41f3-8d08-f5a9759b8bea"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:30:19.989483 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.989460    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-oauth-serving-cert" pod "ed04f535-aa10-41f3-8d08-f5a9759b8bea" (UID: "ed04f535-aa10-41f3-8d08-f5a9759b8bea"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:30:19.990665 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.990643    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-oauth-config" pod "ed04f535-aa10-41f3-8d08-f5a9759b8bea" (UID: "ed04f535-aa10-41f3-8d08-f5a9759b8bea"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:30:19.990784 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.990681    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-serving-cert" pod "ed04f535-aa10-41f3-8d08-f5a9759b8bea" (UID: "ed04f535-aa10-41f3-8d08-f5a9759b8bea"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:30:19.990784 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:19.990772    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed04f535-aa10-41f3-8d08-f5a9759b8bea-kube-api-access-sxrtq" pod "ed04f535-aa10-41f3-8d08-f5a9759b8bea" (UID: "ed04f535-aa10-41f3-8d08-f5a9759b8bea"). InnerVolumeSpecName "kube-api-access-sxrtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:30:20.090491 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.090427    4322 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:30:20.090491 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.090448    4322 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-service-ca\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:30:20.090491 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.090458    4322 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-oauth-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:30:20.090491 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.090467    4322 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed04f535-aa10-41f3-8d08-f5a9759b8bea-console-serving-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:30:20.090491 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.090475    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxrtq\" (UniqueName: \"kubernetes.io/projected/ed04f535-aa10-41f3-8d08-f5a9759b8bea-kube-api-access-sxrtq\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:30:20.090491 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.090484    4322 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed04f535-aa10-41f3-8d08-f5a9759b8bea-oauth-serving-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:30:20.174880 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.174848    4322 generic.go:356] "Generic (PLEG): container finished" podID="ed04f535-aa10-41f3-8d08-f5a9759b8bea" containerID="d60a785ccbc61af7dbaddbb16914249dbdbf2d75fc8b8e0f839e509a64d4c6ef" exitCode=0
Oct 31 07:30:20.175036 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.174898    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69496b9db8-5vsmv"
Oct 31 07:30:20.175036 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.174931    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69496b9db8-5vsmv" event={"ID":"ed04f535-aa10-41f3-8d08-f5a9759b8bea","Type":"ContainerDied","Data":"d60a785ccbc61af7dbaddbb16914249dbdbf2d75fc8b8e0f839e509a64d4c6ef"}
Oct 31 07:30:20.175036 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.174981    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69496b9db8-5vsmv" event={"ID":"ed04f535-aa10-41f3-8d08-f5a9759b8bea","Type":"ContainerDied","Data":"546d631c5686bdd5e21c4eb60dcb9545b06a3122fccab02db174e507f76f388c"}
Oct 31 07:30:20.175036 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.175002    4322 scope.go:122] "RemoveContainer" containerID="d60a785ccbc61af7dbaddbb16914249dbdbf2d75fc8b8e0f839e509a64d4c6ef"
Oct 31 07:30:20.183513 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.183496    4322 scope.go:122] "RemoveContainer" containerID="d60a785ccbc61af7dbaddbb16914249dbdbf2d75fc8b8e0f839e509a64d4c6ef"
Oct 31 07:30:20.183781 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:30:20.183764    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d60a785ccbc61af7dbaddbb16914249dbdbf2d75fc8b8e0f839e509a64d4c6ef\": container with ID starting with d60a785ccbc61af7dbaddbb16914249dbdbf2d75fc8b8e0f839e509a64d4c6ef not found: ID does not exist" containerID="d60a785ccbc61af7dbaddbb16914249dbdbf2d75fc8b8e0f839e509a64d4c6ef"
Oct 31 07:30:20.183847 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.183793    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d60a785ccbc61af7dbaddbb16914249dbdbf2d75fc8b8e0f839e509a64d4c6ef"} err="failed to get container status \"d60a785ccbc61af7dbaddbb16914249dbdbf2d75fc8b8e0f839e509a64d4c6ef\": rpc error: code = NotFound desc = could not find container \"d60a785ccbc61af7dbaddbb16914249dbdbf2d75fc8b8e0f839e509a64d4c6ef\": container with ID starting with d60a785ccbc61af7dbaddbb16914249dbdbf2d75fc8b8e0f839e509a64d4c6ef not found: ID does not exist"
Oct 31 07:30:20.223270 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.223247    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69496b9db8-5vsmv"]
Oct 31 07:30:20.227269 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.227249    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69496b9db8-5vsmv"]
Oct 31 07:30:20.874008 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:20.870770    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed04f535-aa10-41f3-8d08-f5a9759b8bea" path="/var/lib/kubelet/pods/ed04f535-aa10-41f3-8d08-f5a9759b8bea/volumes"
Oct 31 07:30:23.994105 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:23.994071    4322 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:30:23.997884 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:23.997863    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5b647558c9-hg4hd"
Oct 31 07:30:28.493061 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.493028    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dfcbd977f-lr5p9"]
Oct 31 07:30:28.493513 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.493330    4322 cpu_manager.go:414] "containerMap: removing container" podUID="ed04f535-aa10-41f3-8d08-f5a9759b8bea" containerName="console"
Oct 31 07:30:28.493513 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.493341    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed04f535-aa10-41f3-8d08-f5a9759b8bea" containerName="console"
Oct 31 07:30:28.493513 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.493396    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed04f535-aa10-41f3-8d08-f5a9759b8bea" containerName="console"
Oct 31 07:30:28.500345 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.500318    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.514706 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.514684    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 31 07:30:28.526568 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.526545    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 31 07:30:28.527025 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.527007    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 31 07:30:28.527187 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.527169    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 31 07:30:28.538759 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.538741    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 31 07:30:28.539343 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.539326    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 31 07:30:28.539343 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.539326    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 31 07:30:28.551380 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.551360    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pnf7c\""
Oct 31 07:30:28.551907 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.551886    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 31 07:30:28.552992 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.552972    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dfcbd977f-lr5p9"]
Oct 31 07:30:28.660179 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.660154    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8156d449-5d27-40c8-b65c-58394da0fad3-console-oauth-config\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.660304 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.660188    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8156d449-5d27-40c8-b65c-58394da0fad3-service-ca\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.660304 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.660204    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8156d449-5d27-40c8-b65c-58394da0fad3-oauth-serving-cert\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.660304 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.660223    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8156d449-5d27-40c8-b65c-58394da0fad3-console-config\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.660411 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.660317    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th99t\" (UniqueName: \"kubernetes.io/projected/8156d449-5d27-40c8-b65c-58394da0fad3-kube-api-access-th99t\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.660411 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.660348    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8156d449-5d27-40c8-b65c-58394da0fad3-console-serving-cert\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.660411 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.660366    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8156d449-5d27-40c8-b65c-58394da0fad3-trusted-ca-bundle\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.660411 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.660396    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8156d449-5d27-40c8-b65c-58394da0fad3-tmp\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.761527 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.761502    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8156d449-5d27-40c8-b65c-58394da0fad3-service-ca\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.761672 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.761538    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8156d449-5d27-40c8-b65c-58394da0fad3-oauth-serving-cert\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.761672 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.761562    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8156d449-5d27-40c8-b65c-58394da0fad3-console-config\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.761672 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.761584    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th99t\" (UniqueName: \"kubernetes.io/projected/8156d449-5d27-40c8-b65c-58394da0fad3-kube-api-access-th99t\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.761672 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.761609    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8156d449-5d27-40c8-b65c-58394da0fad3-console-serving-cert\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.761917 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.761723    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8156d449-5d27-40c8-b65c-58394da0fad3-trusted-ca-bundle\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.761917 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.761787    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8156d449-5d27-40c8-b65c-58394da0fad3-tmp\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.761917 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.761883    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8156d449-5d27-40c8-b65c-58394da0fad3-console-oauth-config\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.762252 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.762207    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8156d449-5d27-40c8-b65c-58394da0fad3-tmp\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.762380 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.762357    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8156d449-5d27-40c8-b65c-58394da0fad3-service-ca\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.762446 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.762376    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8156d449-5d27-40c8-b65c-58394da0fad3-console-config\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.762446 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.762428    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8156d449-5d27-40c8-b65c-58394da0fad3-oauth-serving-cert\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.762554 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.762506    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8156d449-5d27-40c8-b65c-58394da0fad3-trusted-ca-bundle\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.764071 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.764052    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8156d449-5d27-40c8-b65c-58394da0fad3-console-serving-cert\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.764183 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.764166    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8156d449-5d27-40c8-b65c-58394da0fad3-console-oauth-config\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.784361 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.784330    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-th99t\" (UniqueName: \"kubernetes.io/projected/8156d449-5d27-40c8-b65c-58394da0fad3-kube-api-access-th99t\") pod \"console-7dfcbd977f-lr5p9\" (UID: \"8156d449-5d27-40c8-b65c-58394da0fad3\") " pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:28.810856 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:28.810836    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:29.102455 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:29.102366    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dfcbd977f-lr5p9"]
Oct 31 07:30:29.106056 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:30:29.106020    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8156d449_5d27_40c8_b65c_58394da0fad3.slice/crio-407500c670b50dc8698379d8e3ac814234b028835b7b6dcecba6455fe0853cb0 WatchSource:0}: Error finding container 407500c670b50dc8698379d8e3ac814234b028835b7b6dcecba6455fe0853cb0: Status 404 returned error can't find the container with id 407500c670b50dc8698379d8e3ac814234b028835b7b6dcecba6455fe0853cb0
Oct 31 07:30:29.208413 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:29.208380    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dfcbd977f-lr5p9" event={"ID":"8156d449-5d27-40c8-b65c-58394da0fad3","Type":"ContainerStarted","Data":"5c9f9898f7e92241459f3fd81b47608b24b61c6b9e306ce1c037fbbb25c40eae"}
Oct 31 07:30:29.208561 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:29.208456    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dfcbd977f-lr5p9" event={"ID":"8156d449-5d27-40c8-b65c-58394da0fad3","Type":"ContainerStarted","Data":"407500c670b50dc8698379d8e3ac814234b028835b7b6dcecba6455fe0853cb0"}
Oct 31 07:30:29.250629 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:29.250578    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7dfcbd977f-lr5p9" podStartSLOduration=1.250564657 podStartE2EDuration="1.250564657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:30:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:29.250208487 +0000 UTC m=+70.514029098" watchObservedRunningTime="2026-10-31 07:30:29.250564657 +0000 UTC m=+70.514385267"
Oct 31 07:30:31.709589 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:30:31.709549    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:30:31.915179 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.915144    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 31 07:30:31.918865 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.918848    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:31.933860 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.933839    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 31 07:30:31.933968 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.933953    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 31 07:30:31.945876 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.945849    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 31 07:30:31.945997 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.945892    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 31 07:30:31.945997 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.945948    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-cbt5r\""
Oct 31 07:30:31.959284 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.958618    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 31 07:30:31.959284 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.958635    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 31 07:30:31.959284 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.958659    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 31 07:30:31.959284 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.958665    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 31 07:30:31.959284 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.958688    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 31 07:30:31.959905 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.959867    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 31 07:30:31.985027 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.985007    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfdccab0-b4dc-474b-8300-56269ad1bf4e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:31.985132 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.985033    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:31.985132 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.985064    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:31.985132 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.985109    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-web-config\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:31.985251 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.985163    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfdccab0-b4dc-474b-8300-56269ad1bf4e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:31.985251 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.985185    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfdccab0-b4dc-474b-8300-56269ad1bf4e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:31.985251 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.985211    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfdccab0-b4dc-474b-8300-56269ad1bf4e-config-out\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:31.985344 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.985293    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cfdccab0-b4dc-474b-8300-56269ad1bf4e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:31.985344 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.985314    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8f67\" (UniqueName: \"kubernetes.io/projected/cfdccab0-b4dc-474b-8300-56269ad1bf4e-kube-api-access-g8f67\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:31.985344 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.985340    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:31.985436 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.985356    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-config-volume\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:31.985483 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.985470    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:31.985517 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:31.985492    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.086576 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.086547    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfdccab0-b4dc-474b-8300-56269ad1bf4e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.086576 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.086586    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.086818 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.086632    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.086818 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.086656    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-web-config\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.086818 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.086690    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfdccab0-b4dc-474b-8300-56269ad1bf4e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.086818 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.086713    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfdccab0-b4dc-474b-8300-56269ad1bf4e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.086818 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.086735    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfdccab0-b4dc-474b-8300-56269ad1bf4e-config-out\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.086818 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.086788    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cfdccab0-b4dc-474b-8300-56269ad1bf4e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.087111 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.086821    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8f67\" (UniqueName: \"kubernetes.io/projected/cfdccab0-b4dc-474b-8300-56269ad1bf4e-kube-api-access-g8f67\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.087111 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.086865    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.087111 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.086892    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-config-volume\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.087111 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.086941    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.087111 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.086973    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.087613 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.087582    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cfdccab0-b4dc-474b-8300-56269ad1bf4e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.087613 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.087602    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfdccab0-b4dc-474b-8300-56269ad1bf4e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.088013 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.087983    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfdccab0-b4dc-474b-8300-56269ad1bf4e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.089613 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.089585    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfdccab0-b4dc-474b-8300-56269ad1bf4e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.090454 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.089858    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-web-config\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.090454 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.090062    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.090618 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.090481    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.090618 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.090586    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-config-volume\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.090618 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.090602    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.090837 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.090815    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.090885 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.090869    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cfdccab0-b4dc-474b-8300-56269ad1bf4e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.091566 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.091551    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfdccab0-b4dc-474b-8300-56269ad1bf4e-config-out\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.110165 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.110136    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8f67\" (UniqueName: \"kubernetes.io/projected/cfdccab0-b4dc-474b-8300-56269ad1bf4e-kube-api-access-g8f67\") pod \"alertmanager-main-0\" (UID: \"cfdccab0-b4dc-474b-8300-56269ad1bf4e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.246067 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.246001    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 31 07:30:32.606024 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:32.605977    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 31 07:30:32.608794 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:30:32.608767    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfdccab0_b4dc_474b_8300_56269ad1bf4e.slice/crio-29e1822f1915d8313f5f43ef65afe1a4389b8f1ddea74386b3f7b7bbf8e5363b WatchSource:0}: Error finding container 29e1822f1915d8313f5f43ef65afe1a4389b8f1ddea74386b3f7b7bbf8e5363b: Status 404 returned error can't find the container with id 29e1822f1915d8313f5f43ef65afe1a4389b8f1ddea74386b3f7b7bbf8e5363b
Oct 31 07:30:33.224358 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:33.224318    4322 generic.go:356] "Generic (PLEG): container finished" podID="cfdccab0-b4dc-474b-8300-56269ad1bf4e" containerID="3694f045f60e74e9d779a5a6792ad3aa422db9836ce76297f8b4512267c3919a" exitCode=0
Oct 31 07:30:33.224739 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:33.224412    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cfdccab0-b4dc-474b-8300-56269ad1bf4e","Type":"ContainerDied","Data":"3694f045f60e74e9d779a5a6792ad3aa422db9836ce76297f8b4512267c3919a"}
Oct 31 07:30:33.224739 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:33.224481    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cfdccab0-b4dc-474b-8300-56269ad1bf4e","Type":"ContainerStarted","Data":"29e1822f1915d8313f5f43ef65afe1a4389b8f1ddea74386b3f7b7bbf8e5363b"}
Oct 31 07:30:35.238107 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:35.238070    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cfdccab0-b4dc-474b-8300-56269ad1bf4e","Type":"ContainerStarted","Data":"250ff5c5480fc3e5b74d2ffc02bcb0e2d13623e9351e5cf0b8980d277457f637"}
Oct 31 07:30:35.238593 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:35.238151    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cfdccab0-b4dc-474b-8300-56269ad1bf4e","Type":"ContainerStarted","Data":"8af1a1950e4e0d8fedd586b6b125e07995ff124a63816cc19d09e76c4d27fd48"}
Oct 31 07:30:35.238593 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:35.238192    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cfdccab0-b4dc-474b-8300-56269ad1bf4e","Type":"ContainerStarted","Data":"aa87b91b6bcf8e19bed4d5e4316ca90d2c0e5daa4d9888697fd4b6cebd3a53e5"}
Oct 31 07:30:35.238593 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:35.238272    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cfdccab0-b4dc-474b-8300-56269ad1bf4e","Type":"ContainerStarted","Data":"cda1275a87027211592ed3344abf1ebd008297ad7e086005479f57dbeaafb8ae"}
Oct 31 07:30:35.238968 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:35.238946    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cfdccab0-b4dc-474b-8300-56269ad1bf4e","Type":"ContainerStarted","Data":"7fb33de1eb2811cbe92ca82d418eab902714ba33974f7d5036e55063cb45fc19"}
Oct 31 07:30:35.239080 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:35.239015    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cfdccab0-b4dc-474b-8300-56269ad1bf4e","Type":"ContainerStarted","Data":"9aadbfff8f52c470dfa9ea53ca8a86289aa4fea8e0f47b9b35daad84ffdc7bba"}
Oct 31 07:30:35.328860 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:35.328739    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.7263270950000003 podStartE2EDuration="4.32871717s" totalImagesPullingTime="1.591307135s" totalInitContainerRuntime="11.08294ms" isStatefulPod=false podCreationTimestamp="2026-10-31 07:30:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:35.315362572 +0000 UTC m=+76.579183185" watchObservedRunningTime="2026-10-31 07:30:35.32871717 +0000 UTC m=+76.592537784"
Oct 31 07:30:38.811153 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:38.811112    4322 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:38.811579 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:38.811244    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:38.816448 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:38.816426    4322 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:39.257515 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:39.257443    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dfcbd977f-lr5p9"
Oct 31 07:30:40.258854 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:40.258822    4322 generic.go:356] "Generic (PLEG): container finished" podID="deac7042-7310-4a66-98b3-88fbebb4f2f9" containerID="6e478c628c8cb2bbcac22ce4eeceafc962c8f873266a3b0aad09838d25225927" exitCode=0
Oct 31 07:30:40.259260 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:40.258897    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-k76m7-4srkl" event={"ID":"deac7042-7310-4a66-98b3-88fbebb4f2f9","Type":"ContainerDied","Data":"6e478c628c8cb2bbcac22ce4eeceafc962c8f873266a3b0aad09838d25225927"}
Oct 31 07:30:41.388590 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:41.388567    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:30:41.465458 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:41.465419    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-service-ca-bundle\") pod \"deac7042-7310-4a66-98b3-88fbebb4f2f9\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") "
Oct 31 07:30:41.465649 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:41.465516    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-trusted-ca-bundle\") pod \"deac7042-7310-4a66-98b3-88fbebb4f2f9\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") "
Oct 31 07:30:41.465649 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:41.465565    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deac7042-7310-4a66-98b3-88fbebb4f2f9-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/deac7042-7310-4a66-98b3-88fbebb4f2f9-archives-path\") pod \"deac7042-7310-4a66-98b3-88fbebb4f2f9\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") "
Oct 31 07:30:41.465649 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:41.465592    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/deac7042-7310-4a66-98b3-88fbebb4f2f9-kube-api-access-pcn6k\" (UniqueName: \"kubernetes.io/projected/deac7042-7310-4a66-98b3-88fbebb4f2f9-kube-api-access-pcn6k\") pod \"deac7042-7310-4a66-98b3-88fbebb4f2f9\" (UID: \"deac7042-7310-4a66-98b3-88fbebb4f2f9\") "
Oct 31 07:30:41.465885 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:41.465856    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-service-ca-bundle" pod "deac7042-7310-4a66-98b3-88fbebb4f2f9" (UID: "deac7042-7310-4a66-98b3-88fbebb4f2f9"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:30:41.465923 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:41.465883    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deac7042-7310-4a66-98b3-88fbebb4f2f9-archives-path" pod "deac7042-7310-4a66-98b3-88fbebb4f2f9" (UID: "deac7042-7310-4a66-98b3-88fbebb4f2f9"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:30:41.465992 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:41.465968    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-trusted-ca-bundle" pod "deac7042-7310-4a66-98b3-88fbebb4f2f9" (UID: "deac7042-7310-4a66-98b3-88fbebb4f2f9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:30:41.467877 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:41.467859    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deac7042-7310-4a66-98b3-88fbebb4f2f9-kube-api-access-pcn6k" pod "deac7042-7310-4a66-98b3-88fbebb4f2f9" (UID: "deac7042-7310-4a66-98b3-88fbebb4f2f9"). InnerVolumeSpecName "kube-api-access-pcn6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:30:41.566428 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:41.566331    4322 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-service-ca-bundle\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:30:41.566428 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:41.566376    4322 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deac7042-7310-4a66-98b3-88fbebb4f2f9-trusted-ca-bundle\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:30:41.566428 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:41.566389    4322 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/deac7042-7310-4a66-98b3-88fbebb4f2f9-archives-path\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:30:41.566428 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:41.566400    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcn6k\" (UniqueName: \"kubernetes.io/projected/deac7042-7310-4a66-98b3-88fbebb4f2f9-kube-api-access-pcn6k\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:30:42.269456 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:42.269427    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-k76m7-4srkl"
Oct 31 07:30:42.269456 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:42.269449    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-k76m7-4srkl" event={"ID":"deac7042-7310-4a66-98b3-88fbebb4f2f9","Type":"ContainerDied","Data":"04a7d71a2f31142041e2647831fe00ffed120ad1b4103003e326b8698aee6fa0"}
Oct 31 07:30:42.269739 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:30:42.269486    4322 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04a7d71a2f31142041e2647831fe00ffed120ad1b4103003e326b8698aee6fa0"
Oct 31 07:31:01.709862 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:31:01.709824    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:31:31.710387 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:31:31.710311    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:32:01.710211 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:32:01.710172    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:32:31.709694 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:32:31.709660    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:33:01.710947 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:33:01.710870    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:33:17.165262 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.165208    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"]
Oct 31 07:33:17.165741 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.165718    4322 cpu_manager.go:414] "containerMap: removing container" podUID="deac7042-7310-4a66-98b3-88fbebb4f2f9" containerName="insights-gathering"
Oct 31 07:33:17.165741 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.165738    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="deac7042-7310-4a66-98b3-88fbebb4f2f9" containerName="insights-gathering"
Oct 31 07:33:17.165854 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.165818    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="deac7042-7310-4a66-98b3-88fbebb4f2f9" containerName="insights-gathering"
Oct 31 07:33:17.168936 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.168913    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"
Oct 31 07:33:17.183667 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.183645    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Oct 31 07:33:17.183796 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.183780    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Oct 31 07:33:17.195774 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.195751    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Oct 31 07:33:17.195895 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.195827    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Oct 31 07:33:17.197109 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.197084    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"]
Oct 31 07:33:17.225071 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.225039    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx9tv\" (UniqueName: \"kubernetes.io/projected/c346d169-eb48-412d-b9c0-d2d2fa60600e-kube-api-access-nx9tv\") pod \"keda-metrics-apiserver-86bd856cfb-x9clj\" (UID: \"c346d169-eb48-412d-b9c0-d2d2fa60600e\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"
Oct 31 07:33:17.225243 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.225155    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c346d169-eb48-412d-b9c0-d2d2fa60600e-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-x9clj\" (UID: \"c346d169-eb48-412d-b9c0-d2d2fa60600e\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"
Oct 31 07:33:17.225243 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.225212    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/c346d169-eb48-412d-b9c0-d2d2fa60600e-temp-vol\") pod \"keda-metrics-apiserver-86bd856cfb-x9clj\" (UID: \"c346d169-eb48-412d-b9c0-d2d2fa60600e\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"
Oct 31 07:33:17.285181 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.285143    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Oct 31 07:33:17.285402 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.285307    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-kd9hj\""
Oct 31 07:33:17.325820 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.325781    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c346d169-eb48-412d-b9c0-d2d2fa60600e-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-x9clj\" (UID: \"c346d169-eb48-412d-b9c0-d2d2fa60600e\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"
Oct 31 07:33:17.326013 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.325846    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/c346d169-eb48-412d-b9c0-d2d2fa60600e-temp-vol\") pod \"keda-metrics-apiserver-86bd856cfb-x9clj\" (UID: \"c346d169-eb48-412d-b9c0-d2d2fa60600e\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"
Oct 31 07:33:17.326013 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.325882    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx9tv\" (UniqueName: \"kubernetes.io/projected/c346d169-eb48-412d-b9c0-d2d2fa60600e-kube-api-access-nx9tv\") pod \"keda-metrics-apiserver-86bd856cfb-x9clj\" (UID: \"c346d169-eb48-412d-b9c0-d2d2fa60600e\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"
Oct 31 07:33:17.326282 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.326257    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/c346d169-eb48-412d-b9c0-d2d2fa60600e-temp-vol\") pod \"keda-metrics-apiserver-86bd856cfb-x9clj\" (UID: \"c346d169-eb48-412d-b9c0-d2d2fa60600e\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"
Oct 31 07:33:17.328446 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.328422    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c346d169-eb48-412d-b9c0-d2d2fa60600e-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-x9clj\" (UID: \"c346d169-eb48-412d-b9c0-d2d2fa60600e\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"
Oct 31 07:33:17.348923 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.348894    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx9tv\" (UniqueName: \"kubernetes.io/projected/c346d169-eb48-412d-b9c0-d2d2fa60600e-kube-api-access-nx9tv\") pod \"keda-metrics-apiserver-86bd856cfb-x9clj\" (UID: \"c346d169-eb48-412d-b9c0-d2d2fa60600e\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"
Oct 31 07:33:17.462723 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.462648    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-7b7c985c58-vtzt9"]
Oct 31 07:33:17.466174 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.466155    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-7b7c985c58-vtzt9"
Oct 31 07:33:17.480920 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.480899    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"
Oct 31 07:33:17.485183 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.485164    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Oct 31 07:33:17.486458 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.486436    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-7b7c985c58-vtzt9"]
Oct 31 07:33:17.526997 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.526965    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9j6z\" (UniqueName: \"kubernetes.io/projected/f7d0742a-ca04-480a-ae4b-fbadb2d6cf6c-kube-api-access-r9j6z\") pod \"keda-admission-7b7c985c58-vtzt9\" (UID: \"f7d0742a-ca04-480a-ae4b-fbadb2d6cf6c\") " pod="openshift-keda/keda-admission-7b7c985c58-vtzt9"
Oct 31 07:33:17.527324 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.527298    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/f7d0742a-ca04-480a-ae4b-fbadb2d6cf6c-certificates\") pod \"keda-admission-7b7c985c58-vtzt9\" (UID: \"f7d0742a-ca04-480a-ae4b-fbadb2d6cf6c\") " pod="openshift-keda/keda-admission-7b7c985c58-vtzt9"
Oct 31 07:33:17.628092 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.628055    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9j6z\" (UniqueName: \"kubernetes.io/projected/f7d0742a-ca04-480a-ae4b-fbadb2d6cf6c-kube-api-access-r9j6z\") pod \"keda-admission-7b7c985c58-vtzt9\" (UID: \"f7d0742a-ca04-480a-ae4b-fbadb2d6cf6c\") " pod="openshift-keda/keda-admission-7b7c985c58-vtzt9"
Oct 31 07:33:17.628298 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.628120    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/f7d0742a-ca04-480a-ae4b-fbadb2d6cf6c-certificates\") pod \"keda-admission-7b7c985c58-vtzt9\" (UID: \"f7d0742a-ca04-480a-ae4b-fbadb2d6cf6c\") " pod="openshift-keda/keda-admission-7b7c985c58-vtzt9"
Oct 31 07:33:17.631815 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.631789    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/f7d0742a-ca04-480a-ae4b-fbadb2d6cf6c-certificates\") pod \"keda-admission-7b7c985c58-vtzt9\" (UID: \"f7d0742a-ca04-480a-ae4b-fbadb2d6cf6c\") " pod="openshift-keda/keda-admission-7b7c985c58-vtzt9"
Oct 31 07:33:17.651476 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.651446    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9j6z\" (UniqueName: \"kubernetes.io/projected/f7d0742a-ca04-480a-ae4b-fbadb2d6cf6c-kube-api-access-r9j6z\") pod \"keda-admission-7b7c985c58-vtzt9\" (UID: \"f7d0742a-ca04-480a-ae4b-fbadb2d6cf6c\") " pod="openshift-keda/keda-admission-7b7c985c58-vtzt9"
Oct 31 07:33:17.777933 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.777905    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"]
Oct 31 07:33:17.778792 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.778742    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-7b7c985c58-vtzt9"
Oct 31 07:33:17.781735 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:33:17.781702    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc346d169_eb48_412d_b9c0_d2d2fa60600e.slice/crio-0ab8d901d88cf4206854a9cc5c0819e8448828d82424516647347bc220523d42 WatchSource:0}: Error finding container 0ab8d901d88cf4206854a9cc5c0819e8448828d82424516647347bc220523d42: Status 404 returned error can't find the container with id 0ab8d901d88cf4206854a9cc5c0819e8448828d82424516647347bc220523d42
Oct 31 07:33:17.825779 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:17.825743    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj" event={"ID":"c346d169-eb48-412d-b9c0-d2d2fa60600e","Type":"ContainerStarted","Data":"0ab8d901d88cf4206854a9cc5c0819e8448828d82424516647347bc220523d42"}
Oct 31 07:33:18.076280 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:18.076184    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-7b7c985c58-vtzt9"]
Oct 31 07:33:18.079344 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:33:18.079315    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7d0742a_ca04_480a_ae4b_fbadb2d6cf6c.slice/crio-a36b77282832f538256f9c84f69aa6b05c972520e452a4000a253f0f92d90c17 WatchSource:0}: Error finding container a36b77282832f538256f9c84f69aa6b05c972520e452a4000a253f0f92d90c17: Status 404 returned error can't find the container with id a36b77282832f538256f9c84f69aa6b05c972520e452a4000a253f0f92d90c17
Oct 31 07:33:18.835077 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:18.835013    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-7b7c985c58-vtzt9" event={"ID":"f7d0742a-ca04-480a-ae4b-fbadb2d6cf6c","Type":"ContainerStarted","Data":"a36b77282832f538256f9c84f69aa6b05c972520e452a4000a253f0f92d90c17"}
Oct 31 07:33:21.850560 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:21.850522    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj" event={"ID":"c346d169-eb48-412d-b9c0-d2d2fa60600e","Type":"ContainerStarted","Data":"acc10cbe222144128006bee7fb9f6e3cd339756e87fd29b76c5c4f0be8a17760"}
Oct 31 07:33:21.851023 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:21.850642    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"
Oct 31 07:33:21.852195 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:21.852168    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-7b7c985c58-vtzt9" event={"ID":"f7d0742a-ca04-480a-ae4b-fbadb2d6cf6c","Type":"ContainerStarted","Data":"1c17d8cf5952f01bc82b0eb9419e0f639359c755d36c02442da56a87d8c0a2c0"}
Oct 31 07:33:21.852389 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:21.852370    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-7b7c985c58-vtzt9"
Oct 31 07:33:21.892892 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:21.892827    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj" podStartSLOduration=1.524249094 podStartE2EDuration="4.892810272s" totalImagesPullingTime="3.368561178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:33:21.891852579 +0000 UTC m=+243.155673188" watchObservedRunningTime="2026-10-31 07:33:21.892810272 +0000 UTC m=+243.156630886"
Oct 31 07:33:21.933291 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:21.933245    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-keda/keda-admission-7b7c985c58-vtzt9" podStartSLOduration=2.1204440509999998 podStartE2EDuration="4.933212307s" totalImagesPullingTime="2.812768256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:33:21.932122413 +0000 UTC m=+243.195943036" watchObservedRunningTime="2026-10-31 07:33:21.933212307 +0000 UTC m=+243.197032917"
Oct 31 07:33:31.711474 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:33:31.711423    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:33:32.861554 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:32.861523    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-x9clj"
Oct 31 07:33:42.858438 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:33:42.858410    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-7b7c985c58-vtzt9"
Oct 31 07:34:01.709651 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:34:01.709613    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:34:31.711916 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:34:31.711812    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:34:43.373287 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.373251    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-78bb9c6ff8-mqtfz"]
Oct 31 07:34:43.377010 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.376987    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-78bb9c6ff8-mqtfz"
Oct 31 07:34:43.396988 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.396966    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-78bb9c6ff8-mqtfz"]
Oct 31 07:34:43.418058 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.418029    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Oct 31 07:34:43.418161 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.418090    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-4pz54\""
Oct 31 07:34:43.429698 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.429680    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 31 07:34:43.429792 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.429720    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 31 07:34:43.560714 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.560685    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d478f24d-22f1-4858-8ade-f50ec41e88d6-data\") pod \"seaweedfs-78bb9c6ff8-mqtfz\" (UID: \"d478f24d-22f1-4858-8ade-f50ec41e88d6\") " pod="kserve/seaweedfs-78bb9c6ff8-mqtfz"
Oct 31 07:34:43.560845 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.560728    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x6t6\" (UniqueName: \"kubernetes.io/projected/d478f24d-22f1-4858-8ade-f50ec41e88d6-kube-api-access-8x6t6\") pod \"seaweedfs-78bb9c6ff8-mqtfz\" (UID: \"d478f24d-22f1-4858-8ade-f50ec41e88d6\") " pod="kserve/seaweedfs-78bb9c6ff8-mqtfz"
Oct 31 07:34:43.662073 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.662007    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d478f24d-22f1-4858-8ade-f50ec41e88d6-data\") pod \"seaweedfs-78bb9c6ff8-mqtfz\" (UID: \"d478f24d-22f1-4858-8ade-f50ec41e88d6\") " pod="kserve/seaweedfs-78bb9c6ff8-mqtfz"
Oct 31 07:34:43.662073 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.662051    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x6t6\" (UniqueName: \"kubernetes.io/projected/d478f24d-22f1-4858-8ade-f50ec41e88d6-kube-api-access-8x6t6\") pod \"seaweedfs-78bb9c6ff8-mqtfz\" (UID: \"d478f24d-22f1-4858-8ade-f50ec41e88d6\") " pod="kserve/seaweedfs-78bb9c6ff8-mqtfz"
Oct 31 07:34:43.662446 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.662428    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d478f24d-22f1-4858-8ade-f50ec41e88d6-data\") pod \"seaweedfs-78bb9c6ff8-mqtfz\" (UID: \"d478f24d-22f1-4858-8ade-f50ec41e88d6\") " pod="kserve/seaweedfs-78bb9c6ff8-mqtfz"
Oct 31 07:34:43.684772 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.684748    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x6t6\" (UniqueName: \"kubernetes.io/projected/d478f24d-22f1-4858-8ade-f50ec41e88d6-kube-api-access-8x6t6\") pod \"seaweedfs-78bb9c6ff8-mqtfz\" (UID: \"d478f24d-22f1-4858-8ade-f50ec41e88d6\") " pod="kserve/seaweedfs-78bb9c6ff8-mqtfz"
Oct 31 07:34:43.688562 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.688543    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-78bb9c6ff8-mqtfz"
Oct 31 07:34:43.979660 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.979585    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-78bb9c6ff8-mqtfz"]
Oct 31 07:34:43.983068 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:34:43.983047    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd478f24d_22f1_4858_8ade_f50ec41e88d6.slice/crio-e78aae7d21498d6b27cafef97c4e2fc6acc9920552f8d84f282223736708958d WatchSource:0}: Error finding container e78aae7d21498d6b27cafef97c4e2fc6acc9920552f8d84f282223736708958d: Status 404 returned error can't find the container with id e78aae7d21498d6b27cafef97c4e2fc6acc9920552f8d84f282223736708958d
Oct 31 07:34:43.984741 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:43.984722    4322 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 07:34:44.156081 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:44.156052    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-78bb9c6ff8-mqtfz" event={"ID":"d478f24d-22f1-4858-8ade-f50ec41e88d6","Type":"ContainerStarted","Data":"e78aae7d21498d6b27cafef97c4e2fc6acc9920552f8d84f282223736708958d"}
Oct 31 07:34:47.169593 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:47.169555    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-78bb9c6ff8-mqtfz" event={"ID":"d478f24d-22f1-4858-8ade-f50ec41e88d6","Type":"ContainerStarted","Data":"a54b81f5a28cec57559ba00d57bc5617a7355dc6ee942e73e71b7b21fa0bdc18"}
Oct 31 07:34:47.169948 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:47.169692    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-78bb9c6ff8-mqtfz"
Oct 31 07:34:47.209550 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:47.209493    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/seaweedfs-78bb9c6ff8-mqtfz" podStartSLOduration=1.608672648 podStartE2EDuration="4.209475778s" totalImagesPullingTime="2.60080313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:34:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:34:47.208008294 +0000 UTC m=+328.471828905" watchObservedRunningTime="2026-10-31 07:34:47.209475778 +0000 UTC m=+328.473296390"
Oct 31 07:34:53.178104 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:34:53.178069    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-78bb9c6ff8-mqtfz"
Oct 31 07:35:01.709598 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:35:01.709560    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:35:31.709554 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:35:31.709515    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:36:01.711397 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:36:01.711308    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:36:06.319887 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.319851    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-56ddc4c448-mblfh"]
Oct 31 07:36:06.323580 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.323558    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-56ddc4c448-mblfh"
Oct 31 07:36:06.348316 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.348293    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-56ddc4c448-mblfh"]
Oct 31 07:36:06.351369 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.351349    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-hplcd\""
Oct 31 07:36:06.351605 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.351586    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Oct 31 07:36:06.376598 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.376569    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7f9f8cdc65-bpncd"]
Oct 31 07:36:06.379865 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.379849    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7f9f8cdc65-bpncd"
Oct 31 07:36:06.399263 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.399239    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7f9f8cdc65-bpncd"]
Oct 31 07:36:06.403931 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.403911    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-rr67s\""
Oct 31 07:36:06.404032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.403979    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\""
Oct 31 07:36:06.404032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.404008    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Oct 31 07:36:06.422395 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.422368    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d11d0af-46d6-4103-b835-45910e9c4e90-cert\") pod \"odh-model-controller-7f9f8cdc65-bpncd\" (UID: \"3d11d0af-46d6-4103-b835-45910e9c4e90\") " pod="kserve/odh-model-controller-7f9f8cdc65-bpncd"
Oct 31 07:36:06.422495 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.422479    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d11d0af-46d6-4103-b835-45910e9c4e90-metrics-certs\") pod \"odh-model-controller-7f9f8cdc65-bpncd\" (UID: \"3d11d0af-46d6-4103-b835-45910e9c4e90\") " pod="kserve/odh-model-controller-7f9f8cdc65-bpncd"
Oct 31 07:36:06.422543 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.422531    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e88b03c2-b842-45e4-b116-bf0b744eed6b-tls-certs\") pod \"model-serving-api-56ddc4c448-mblfh\" (UID: \"e88b03c2-b842-45e4-b116-bf0b744eed6b\") " pod="kserve/model-serving-api-56ddc4c448-mblfh"
Oct 31 07:36:06.422589 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.422550    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdhsf\" (UniqueName: \"kubernetes.io/projected/3d11d0af-46d6-4103-b835-45910e9c4e90-kube-api-access-mdhsf\") pod \"odh-model-controller-7f9f8cdc65-bpncd\" (UID: \"3d11d0af-46d6-4103-b835-45910e9c4e90\") " pod="kserve/odh-model-controller-7f9f8cdc65-bpncd"
Oct 31 07:36:06.422589 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.422569    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h52gz\" (UniqueName: \"kubernetes.io/projected/e88b03c2-b842-45e4-b116-bf0b744eed6b-kube-api-access-h52gz\") pod \"model-serving-api-56ddc4c448-mblfh\" (UID: \"e88b03c2-b842-45e4-b116-bf0b744eed6b\") " pod="kserve/model-serving-api-56ddc4c448-mblfh"
Oct 31 07:36:06.523617 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.523585    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d11d0af-46d6-4103-b835-45910e9c4e90-metrics-certs\") pod \"odh-model-controller-7f9f8cdc65-bpncd\" (UID: \"3d11d0af-46d6-4103-b835-45910e9c4e90\") " pod="kserve/odh-model-controller-7f9f8cdc65-bpncd"
Oct 31 07:36:06.523774 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.523634    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e88b03c2-b842-45e4-b116-bf0b744eed6b-tls-certs\") pod \"model-serving-api-56ddc4c448-mblfh\" (UID: \"e88b03c2-b842-45e4-b116-bf0b744eed6b\") " pod="kserve/model-serving-api-56ddc4c448-mblfh"
Oct 31 07:36:06.523774 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.523654    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdhsf\" (UniqueName: \"kubernetes.io/projected/3d11d0af-46d6-4103-b835-45910e9c4e90-kube-api-access-mdhsf\") pod \"odh-model-controller-7f9f8cdc65-bpncd\" (UID: \"3d11d0af-46d6-4103-b835-45910e9c4e90\") " pod="kserve/odh-model-controller-7f9f8cdc65-bpncd"
Oct 31 07:36:06.523774 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.523671    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h52gz\" (UniqueName: \"kubernetes.io/projected/e88b03c2-b842-45e4-b116-bf0b744eed6b-kube-api-access-h52gz\") pod \"model-serving-api-56ddc4c448-mblfh\" (UID: \"e88b03c2-b842-45e4-b116-bf0b744eed6b\") " pod="kserve/model-serving-api-56ddc4c448-mblfh"
Oct 31 07:36:06.523774 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.523702    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d11d0af-46d6-4103-b835-45910e9c4e90-cert\") pod \"odh-model-controller-7f9f8cdc65-bpncd\" (UID: \"3d11d0af-46d6-4103-b835-45910e9c4e90\") " pod="kserve/odh-model-controller-7f9f8cdc65-bpncd"
Oct 31 07:36:06.526284 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.526252    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e88b03c2-b842-45e4-b116-bf0b744eed6b-tls-certs\") pod \"model-serving-api-56ddc4c448-mblfh\" (UID: \"e88b03c2-b842-45e4-b116-bf0b744eed6b\") " pod="kserve/model-serving-api-56ddc4c448-mblfh"
Oct 31 07:36:06.526385 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.526260    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d11d0af-46d6-4103-b835-45910e9c4e90-metrics-certs\") pod \"odh-model-controller-7f9f8cdc65-bpncd\" (UID: \"3d11d0af-46d6-4103-b835-45910e9c4e90\") " pod="kserve/odh-model-controller-7f9f8cdc65-bpncd"
Oct 31 07:36:06.526385 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.526261    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d11d0af-46d6-4103-b835-45910e9c4e90-cert\") pod \"odh-model-controller-7f9f8cdc65-bpncd\" (UID: \"3d11d0af-46d6-4103-b835-45910e9c4e90\") " pod="kserve/odh-model-controller-7f9f8cdc65-bpncd"
Oct 31 07:36:06.546241 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.546203    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h52gz\" (UniqueName: \"kubernetes.io/projected/e88b03c2-b842-45e4-b116-bf0b744eed6b-kube-api-access-h52gz\") pod \"model-serving-api-56ddc4c448-mblfh\" (UID: \"e88b03c2-b842-45e4-b116-bf0b744eed6b\") " pod="kserve/model-serving-api-56ddc4c448-mblfh"
Oct 31 07:36:06.546438 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.546416    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdhsf\" (UniqueName: \"kubernetes.io/projected/3d11d0af-46d6-4103-b835-45910e9c4e90-kube-api-access-mdhsf\") pod \"odh-model-controller-7f9f8cdc65-bpncd\" (UID: \"3d11d0af-46d6-4103-b835-45910e9c4e90\") " pod="kserve/odh-model-controller-7f9f8cdc65-bpncd"
Oct 31 07:36:06.637649 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.637577    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-56ddc4c448-mblfh"
Oct 31 07:36:06.691630 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.691602    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7f9f8cdc65-bpncd"
Oct 31 07:36:06.930747 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.930669    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-56ddc4c448-mblfh"]
Oct 31 07:36:06.933901 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:36:06.933873    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode88b03c2_b842_45e4_b116_bf0b744eed6b.slice/crio-d5a16cc91482b4aa8f35eda8d40b7d516cf9b2f6203205fbbe2c3639353c2fa7 WatchSource:0}: Error finding container d5a16cc91482b4aa8f35eda8d40b7d516cf9b2f6203205fbbe2c3639353c2fa7: Status 404 returned error can't find the container with id d5a16cc91482b4aa8f35eda8d40b7d516cf9b2f6203205fbbe2c3639353c2fa7
Oct 31 07:36:06.984911 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:06.984877    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7f9f8cdc65-bpncd"]
Oct 31 07:36:06.988505 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:36:06.988474    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d11d0af_46d6_4103_b835_45910e9c4e90.slice/crio-93c8fe829211a97cafb6580164d24c635152becbeec422714832ff5e834988ad WatchSource:0}: Error finding container 93c8fe829211a97cafb6580164d24c635152becbeec422714832ff5e834988ad: Status 404 returned error can't find the container with id 93c8fe829211a97cafb6580164d24c635152becbeec422714832ff5e834988ad
Oct 31 07:36:07.478316 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:07.478271    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7f9f8cdc65-bpncd" event={"ID":"3d11d0af-46d6-4103-b835-45910e9c4e90","Type":"ContainerStarted","Data":"93c8fe829211a97cafb6580164d24c635152becbeec422714832ff5e834988ad"}
Oct 31 07:36:07.480633 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:07.480604    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-56ddc4c448-mblfh" event={"ID":"e88b03c2-b842-45e4-b116-bf0b744eed6b","Type":"ContainerStarted","Data":"d5a16cc91482b4aa8f35eda8d40b7d516cf9b2f6203205fbbe2c3639353c2fa7"}
Oct 31 07:36:10.498126 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:10.498086    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7f9f8cdc65-bpncd" event={"ID":"3d11d0af-46d6-4103-b835-45910e9c4e90","Type":"ContainerStarted","Data":"e81ce6f6bbe224cf7b239d2241ee4bb39acae8208008dab1406b57277ed9bdf6"}
Oct 31 07:36:10.498608 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:10.498183    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7f9f8cdc65-bpncd"
Oct 31 07:36:10.499550 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:10.499523    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-56ddc4c448-mblfh" event={"ID":"e88b03c2-b842-45e4-b116-bf0b744eed6b","Type":"ContainerStarted","Data":"3d4d677a74b1b30b864794f29c848ccacaddbcfce32869e9aa232abd2bd47d5a"}
Oct 31 07:36:10.499684 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:10.499654    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-56ddc4c448-mblfh"
Oct 31 07:36:10.538609 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:10.538559    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/odh-model-controller-7f9f8cdc65-bpncd" podStartSLOduration=1.6788972960000001 podStartE2EDuration="4.538546004s" totalImagesPullingTime="2.859648708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:36:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:36:10.537774483 +0000 UTC m=+411.801595098" watchObservedRunningTime="2026-10-31 07:36:10.538546004 +0000 UTC m=+411.802366614"
Oct 31 07:36:10.576938 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:10.576897    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/model-serving-api-56ddc4c448-mblfh" podStartSLOduration=1.897582316 podStartE2EDuration="4.576885188s" totalImagesPullingTime="2.679302872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:36:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:36:10.576820808 +0000 UTC m=+411.840641419" watchObservedRunningTime="2026-10-31 07:36:10.576885188 +0000 UTC m=+411.840705799"
Oct 31 07:36:21.505674 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:21.505639    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7f9f8cdc65-bpncd"
Oct 31 07:36:21.507840 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:21.507819    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-56ddc4c448-mblfh"
Oct 31 07:36:22.898550 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:22.898516    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/s3-init-qt2m7"]
Oct 31 07:36:22.902380 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:22.902359    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-qt2m7"
Oct 31 07:36:22.916494 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:22.916474    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\""
Oct 31 07:36:22.929644 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:22.929618    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-qt2m7"]
Oct 31 07:36:22.967159 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:22.967131    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85da5ec2-23e8-4294-b562-8bbfd30deca6-model-cache\") pod \"s3-init-qt2m7\" (UID: \"85da5ec2-23e8-4294-b562-8bbfd30deca6\") " pod="kserve/s3-init-qt2m7"
Oct 31 07:36:22.967301 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:22.967189    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2gtm\" (UniqueName: \"kubernetes.io/projected/85da5ec2-23e8-4294-b562-8bbfd30deca6-kube-api-access-l2gtm\") pod \"s3-init-qt2m7\" (UID: \"85da5ec2-23e8-4294-b562-8bbfd30deca6\") " pod="kserve/s3-init-qt2m7"
Oct 31 07:36:23.068497 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:23.068470    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85da5ec2-23e8-4294-b562-8bbfd30deca6-model-cache\") pod \"s3-init-qt2m7\" (UID: \"85da5ec2-23e8-4294-b562-8bbfd30deca6\") " pod="kserve/s3-init-qt2m7"
Oct 31 07:36:23.068619 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:23.068520    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2gtm\" (UniqueName: \"kubernetes.io/projected/85da5ec2-23e8-4294-b562-8bbfd30deca6-kube-api-access-l2gtm\") pod \"s3-init-qt2m7\" (UID: \"85da5ec2-23e8-4294-b562-8bbfd30deca6\") " pod="kserve/s3-init-qt2m7"
Oct 31 07:36:23.068818 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:23.068799    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85da5ec2-23e8-4294-b562-8bbfd30deca6-model-cache\") pod \"s3-init-qt2m7\" (UID: \"85da5ec2-23e8-4294-b562-8bbfd30deca6\") " pod="kserve/s3-init-qt2m7"
Oct 31 07:36:23.091492 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:23.091469    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2gtm\" (UniqueName: \"kubernetes.io/projected/85da5ec2-23e8-4294-b562-8bbfd30deca6-kube-api-access-l2gtm\") pod \"s3-init-qt2m7\" (UID: \"85da5ec2-23e8-4294-b562-8bbfd30deca6\") " pod="kserve/s3-init-qt2m7"
Oct 31 07:36:23.222276 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:23.222204    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-qt2m7"
Oct 31 07:36:23.513898 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:23.513870    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-qt2m7"]
Oct 31 07:36:23.517048 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:36:23.517017    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85da5ec2_23e8_4294_b562_8bbfd30deca6.slice/crio-ef6f68495d913752ce1e30aefde44b0875c122e043af3c6680a63f5dd30b9241 WatchSource:0}: Error finding container ef6f68495d913752ce1e30aefde44b0875c122e043af3c6680a63f5dd30b9241: Status 404 returned error can't find the container with id ef6f68495d913752ce1e30aefde44b0875c122e043af3c6680a63f5dd30b9241
Oct 31 07:36:23.554064 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:23.554029    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qt2m7" event={"ID":"85da5ec2-23e8-4294-b562-8bbfd30deca6","Type":"ContainerStarted","Data":"ef6f68495d913752ce1e30aefde44b0875c122e043af3c6680a63f5dd30b9241"}
Oct 31 07:36:27.572966 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:27.572934    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qt2m7" event={"ID":"85da5ec2-23e8-4294-b562-8bbfd30deca6","Type":"ContainerStarted","Data":"37b4ba396c5bb59995978877385cfaee75389a3a2eadf4ae936ca4c90b442ac8"}
Oct 31 07:36:28.578488 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:28.578456    4322 generic.go:356] "Generic (PLEG): container finished" podID="85da5ec2-23e8-4294-b562-8bbfd30deca6" containerID="37b4ba396c5bb59995978877385cfaee75389a3a2eadf4ae936ca4c90b442ac8" exitCode=0
Oct 31 07:36:28.578849 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:28.578524    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qt2m7" event={"ID":"85da5ec2-23e8-4294-b562-8bbfd30deca6","Type":"ContainerDied","Data":"37b4ba396c5bb59995978877385cfaee75389a3a2eadf4ae936ca4c90b442ac8"}
Oct 31 07:36:31.716728 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:36:31.716682    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:36:33.604597 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:33.604540    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qt2m7" event={"ID":"85da5ec2-23e8-4294-b562-8bbfd30deca6","Type":"ContainerStarted","Data":"45829e73a9f097e018d6d44fc96c9d0fffaf23f7a4ee7dc36d74b71187645564"}
Oct 31 07:36:33.645295 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:36:33.645248    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/s3-init-qt2m7" podStartSLOduration=2.845695409 podStartE2EDuration="11.64521382s" totalImagesPullingTime="7.676150823s" totalInitContainerRuntime="1.123367588s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:36:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:36:33.645010306 +0000 UTC m=+434.908830916" watchObservedRunningTime="2026-10-31 07:36:33.64521382 +0000 UTC m=+434.909034433"
Oct 31 07:37:01.710079 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:37:01.710035    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:37:31.710279 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:37:31.710180    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:37:38.870858 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:38.870823    4322 generic.go:356] "Generic (PLEG): container finished" podID="85da5ec2-23e8-4294-b562-8bbfd30deca6" containerID="45829e73a9f097e018d6d44fc96c9d0fffaf23f7a4ee7dc36d74b71187645564" exitCode=0
Oct 31 07:37:38.871307 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:38.870955    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qt2m7" event={"ID":"85da5ec2-23e8-4294-b562-8bbfd30deca6","Type":"ContainerDied","Data":"45829e73a9f097e018d6d44fc96c9d0fffaf23f7a4ee7dc36d74b71187645564"}
Oct 31 07:37:39.994591 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:39.994569    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-qt2m7"
Oct 31 07:37:40.033317 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:40.033291    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85da5ec2-23e8-4294-b562-8bbfd30deca6-kube-api-access-l2gtm\" (UniqueName: \"kubernetes.io/projected/85da5ec2-23e8-4294-b562-8bbfd30deca6-kube-api-access-l2gtm\") pod \"85da5ec2-23e8-4294-b562-8bbfd30deca6\" (UID: \"85da5ec2-23e8-4294-b562-8bbfd30deca6\") "
Oct 31 07:37:40.033446 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:40.033364    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85da5ec2-23e8-4294-b562-8bbfd30deca6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85da5ec2-23e8-4294-b562-8bbfd30deca6-model-cache\") pod \"85da5ec2-23e8-4294-b562-8bbfd30deca6\" (UID: \"85da5ec2-23e8-4294-b562-8bbfd30deca6\") "
Oct 31 07:37:40.035605 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:40.035577    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85da5ec2-23e8-4294-b562-8bbfd30deca6-kube-api-access-l2gtm" pod "85da5ec2-23e8-4294-b562-8bbfd30deca6" (UID: "85da5ec2-23e8-4294-b562-8bbfd30deca6"). InnerVolumeSpecName "kube-api-access-l2gtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:37:40.053096 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:40.053068    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85da5ec2-23e8-4294-b562-8bbfd30deca6-model-cache" pod "85da5ec2-23e8-4294-b562-8bbfd30deca6" (UID: "85da5ec2-23e8-4294-b562-8bbfd30deca6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:37:40.134427 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:40.134357    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2gtm\" (UniqueName: \"kubernetes.io/projected/85da5ec2-23e8-4294-b562-8bbfd30deca6-kube-api-access-l2gtm\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:37:40.134427 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:40.134385    4322 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85da5ec2-23e8-4294-b562-8bbfd30deca6-model-cache\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:37:40.879557 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:40.879529    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-qt2m7" event={"ID":"85da5ec2-23e8-4294-b562-8bbfd30deca6","Type":"ContainerDied","Data":"ef6f68495d913752ce1e30aefde44b0875c122e043af3c6680a63f5dd30b9241"}
Oct 31 07:37:40.879675 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:40.879567    4322 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef6f68495d913752ce1e30aefde44b0875c122e043af3c6680a63f5dd30b9241"
Oct 31 07:37:40.879675 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:40.879575    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-qt2m7"
Oct 31 07:37:58.812879 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.812844    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"]
Oct 31 07:37:58.815609 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.813387    4322 cpu_manager.go:414] "containerMap: removing container" podUID="85da5ec2-23e8-4294-b562-8bbfd30deca6" containerName="s3-init"
Oct 31 07:37:58.815609 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.813410    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85da5ec2-23e8-4294-b562-8bbfd30deca6" containerName="s3-init"
Oct 31 07:37:58.815609 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.813452    4322 cpu_manager.go:414] "containerMap: removing container" podUID="85da5ec2-23e8-4294-b562-8bbfd30deca6" containerName="download-hf-model"
Oct 31 07:37:58.815609 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.813461    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85da5ec2-23e8-4294-b562-8bbfd30deca6" containerName="download-hf-model"
Oct 31 07:37:58.815609 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.813579    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="85da5ec2-23e8-4294-b562-8bbfd30deca6" containerName="s3-init"
Oct 31 07:37:58.818276 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.818255    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:58.833125 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.833103    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"openshift-service-ca.crt\""
Oct 31 07:37:58.844676 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.844656    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 07:37:58.844856 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.844839    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"kube-root-ca.crt\""
Oct 31 07:37:58.845129 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.844849    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\""
Oct 31 07:37:58.845129 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.845019    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7281997251\"/\"isvc-canary-rollback-v2-predictor-serving-cert\""
Oct 31 07:37:58.845129 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.845070    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7281997251\"/\"seaweedfs-s3-creds\""
Oct 31 07:37:58.845764 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.845745    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7281997251\"/\"default-dockercfg-t4zp9\""
Oct 31 07:37:58.847378 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.847357    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"]
Oct 31 07:37:58.893997 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.893967    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/efa5e541-06f0-4ece-a948-5e3037e225de-proxy-tls\") pod \"isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:58.894116 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.894009    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/efa5e541-06f0-4ece-a948-5e3037e225de-cabundle-cert\") pod \"isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:58.894116 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.894045    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/efa5e541-06f0-4ece-a948-5e3037e225de-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:58.894116 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.894108    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlscf\" (UniqueName: \"kubernetes.io/projected/efa5e541-06f0-4ece-a948-5e3037e225de-kube-api-access-rlscf\") pod \"isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:58.894242 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.894185    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/efa5e541-06f0-4ece-a948-5e3037e225de-kserve-provision-location\") pod \"isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:58.995071 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.995041    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/efa5e541-06f0-4ece-a948-5e3037e225de-proxy-tls\") pod \"isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:58.995198 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.995083    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/efa5e541-06f0-4ece-a948-5e3037e225de-cabundle-cert\") pod \"isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:58.995198 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.995114    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/efa5e541-06f0-4ece-a948-5e3037e225de-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:58.995336 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.995260    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlscf\" (UniqueName: \"kubernetes.io/projected/efa5e541-06f0-4ece-a948-5e3037e225de-kube-api-access-rlscf\") pod \"isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:58.995336 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.995313    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/efa5e541-06f0-4ece-a948-5e3037e225de-kserve-provision-location\") pod \"isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:58.995732 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.995707    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/efa5e541-06f0-4ece-a948-5e3037e225de-kserve-provision-location\") pod \"isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:58.995904 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.995886    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/efa5e541-06f0-4ece-a948-5e3037e225de-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:58.995978 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.995914    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/efa5e541-06f0-4ece-a948-5e3037e225de-cabundle-cert\") pod \"isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:58.997613 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:58.997585    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/efa5e541-06f0-4ece-a948-5e3037e225de-proxy-tls\") pod \"isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:59.002031 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.002012    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"]
Oct 31 07:37:59.005956 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.005939    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.017892 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.017873    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlscf\" (UniqueName: \"kubernetes.io/projected/efa5e541-06f0-4ece-a948-5e3037e225de-kube-api-access-rlscf\") pod \"isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:59.020415 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.020393    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7281997251\"/\"isvc-canary-rollback-predictor-serving-cert\""
Oct 31 07:37:59.027562 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.027543    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"]
Oct 31 07:37:59.096578 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.096508    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/023261ef-dadc-4c2e-ad06-5d4c1eba1025-cabundle-cert\") pod \"isvc-canary-rollback-predictor-5b7b487756-wpkkp\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.096578 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.096551    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/023261ef-dadc-4c2e-ad06-5d4c1eba1025-proxy-tls\") pod \"isvc-canary-rollback-predictor-5b7b487756-wpkkp\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.096731 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.096667    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-5b7b487756-wpkkp\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.096731 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.096686    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-5b7b487756-wpkkp\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.096795 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.096727    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxqsv\" (UniqueName: \"kubernetes.io/projected/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kube-api-access-dxqsv\") pod \"isvc-canary-rollback-predictor-5b7b487756-wpkkp\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.137639 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.137614    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:37:59.199392 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.199122    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-5b7b487756-wpkkp\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.199392 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.199176    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-5b7b487756-wpkkp\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.199392 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.199269    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxqsv\" (UniqueName: \"kubernetes.io/projected/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kube-api-access-dxqsv\") pod \"isvc-canary-rollback-predictor-5b7b487756-wpkkp\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.199392 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.199324    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/023261ef-dadc-4c2e-ad06-5d4c1eba1025-cabundle-cert\") pod \"isvc-canary-rollback-predictor-5b7b487756-wpkkp\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.199392 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.199354    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/023261ef-dadc-4c2e-ad06-5d4c1eba1025-proxy-tls\") pod \"isvc-canary-rollback-predictor-5b7b487756-wpkkp\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.199761 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.199737    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-5b7b487756-wpkkp\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.200265 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.200246    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-5b7b487756-wpkkp\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.200615 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.200582    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/023261ef-dadc-4c2e-ad06-5d4c1eba1025-cabundle-cert\") pod \"isvc-canary-rollback-predictor-5b7b487756-wpkkp\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.201878 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.201836    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/023261ef-dadc-4c2e-ad06-5d4c1eba1025-proxy-tls\") pod \"isvc-canary-rollback-predictor-5b7b487756-wpkkp\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.222792 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.222764    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxqsv\" (UniqueName: \"kubernetes.io/projected/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kube-api-access-dxqsv\") pod \"isvc-canary-rollback-predictor-5b7b487756-wpkkp\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.322958 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.322922    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:37:59.445691 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.445664    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"]
Oct 31 07:37:59.448490 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:37:59.448455    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa5e541_06f0_4ece_a948_5e3037e225de.slice/crio-c79b2d15bf1937939e38c63df22f752c7d95cd94b50354a916cf5cb12e2780eb WatchSource:0}: Error finding container c79b2d15bf1937939e38c63df22f752c7d95cd94b50354a916cf5cb12e2780eb: Status 404 returned error can't find the container with id c79b2d15bf1937939e38c63df22f752c7d95cd94b50354a916cf5cb12e2780eb
Oct 31 07:37:59.498117 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.498092    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"]
Oct 31 07:37:59.502183 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.502167    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.519529 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.519502    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"default-dockercfg-d6nzh\""
Oct 31 07:37:59.519638 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.519512    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"openshift-service-ca.crt\""
Oct 31 07:37:59.519638 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.519515    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"seaweedfs-s3-creds\""
Oct 31 07:37:59.519638 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.519535    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"kube-root-ca.crt\""
Oct 31 07:37:59.519638 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.519556    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"isvc-raw-sklearn-batcher-98f08-kube-rbac-proxy-sar-config\""
Oct 31 07:37:59.519638 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.519572    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 07:37:59.519638 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.519577    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"isvc-raw-sklearn-batcher-98f08-predictor-serving-cert\""
Oct 31 07:37:59.524560 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.524538    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"]
Oct 31 07:37:59.602519 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.602484    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0c8980e-1f7e-480b-a633-e7317a35b7e4-proxy-tls\") pod \"isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") " pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.602677 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.602539    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxp6g\" (UniqueName: \"kubernetes.io/projected/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kube-api-access-rxp6g\") pod \"isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") " pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.602738 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.602704    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kserve-provision-location\") pod \"isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") " pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.602786 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.602744    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kube-rbac-proxy-sar-config\") pod \"isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") " pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.602855 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.602827    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f0c8980e-1f7e-480b-a633-e7317a35b7e4-cabundle-cert\") pod \"isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") " pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.619112 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.619022    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"]
Oct 31 07:37:59.621396 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:37:59.621329    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod023261ef_dadc_4c2e_ad06_5d4c1eba1025.slice/crio-a3e79e587362ea2ef86fbed15acd614c456ea896b144879ca506fdbed572a43f WatchSource:0}: Error finding container a3e79e587362ea2ef86fbed15acd614c456ea896b144879ca506fdbed572a43f: Status 404 returned error can't find the container with id a3e79e587362ea2ef86fbed15acd614c456ea896b144879ca506fdbed572a43f
Oct 31 07:37:59.703774 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.703750    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kserve-provision-location\") pod \"isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") " pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.703989 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.703786    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kube-rbac-proxy-sar-config\") pod \"isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") " pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.703989 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.703820    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f0c8980e-1f7e-480b-a633-e7317a35b7e4-cabundle-cert\") pod \"isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") " pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.703989 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.703860    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0c8980e-1f7e-480b-a633-e7317a35b7e4-proxy-tls\") pod \"isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") " pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.703989 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.703899    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxp6g\" (UniqueName: \"kubernetes.io/projected/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kube-api-access-rxp6g\") pod \"isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") " pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.704217 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.704139    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kserve-provision-location\") pod \"isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") " pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.704529 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.704504    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f0c8980e-1f7e-480b-a633-e7317a35b7e4-cabundle-cert\") pod \"isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") " pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.704637 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.704619    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kube-rbac-proxy-sar-config\") pod \"isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") " pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.706289 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.706267    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0c8980e-1f7e-480b-a633-e7317a35b7e4-proxy-tls\") pod \"isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") " pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.726378 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.726345    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxp6g\" (UniqueName: \"kubernetes.io/projected/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kube-api-access-rxp6g\") pod \"isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") " pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.829243 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.829192    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:37:59.958533 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.958494    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" event={"ID":"023261ef-dadc-4c2e-ad06-5d4c1eba1025","Type":"ContainerStarted","Data":"6302dec0319e422a0e1e8fd6b281574e82d4a3a90d6bd87dcbe6bae277cd68fd"}
Oct 31 07:37:59.958701 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.958570    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" event={"ID":"023261ef-dadc-4c2e-ad06-5d4c1eba1025","Type":"ContainerStarted","Data":"a3e79e587362ea2ef86fbed15acd614c456ea896b144879ca506fdbed572a43f"}
Oct 31 07:37:59.960378 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.960350    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" event={"ID":"efa5e541-06f0-4ece-a948-5e3037e225de","Type":"ContainerStarted","Data":"86faed514e5d875234d462ac1dd123821f99d5916d34763e589837c002084769"}
Oct 31 07:37:59.960509 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:37:59.960400    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" event={"ID":"efa5e541-06f0-4ece-a948-5e3037e225de","Type":"ContainerStarted","Data":"c79b2d15bf1937939e38c63df22f752c7d95cd94b50354a916cf5cb12e2780eb"}
Oct 31 07:38:00.137744 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:00.137657    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"]
Oct 31 07:38:00.141409 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:38:00.141369    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0c8980e_1f7e_480b_a633_e7317a35b7e4.slice/crio-0528df177c888156067e976babd8905b9be528fda02cd92b00544ab0c7f05cb2 WatchSource:0}: Error finding container 0528df177c888156067e976babd8905b9be528fda02cd92b00544ab0c7f05cb2: Status 404 returned error can't find the container with id 0528df177c888156067e976babd8905b9be528fda02cd92b00544ab0c7f05cb2
Oct 31 07:38:00.966531 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:00.966489    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" event={"ID":"f0c8980e-1f7e-480b-a633-e7317a35b7e4","Type":"ContainerStarted","Data":"0b855d77eec48c3fc9963475760e651a6bd1ef3f8da5fb0b0992e6459191119a"}
Oct 31 07:38:00.966982 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:00.966556    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" event={"ID":"f0c8980e-1f7e-480b-a633-e7317a35b7e4","Type":"ContainerStarted","Data":"0528df177c888156067e976babd8905b9be528fda02cd92b00544ab0c7f05cb2"}
Oct 31 07:38:01.709514 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:38:01.709475    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:38:03.981295 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:03.981220    4322 generic.go:356] "Generic (PLEG): container finished" podID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerID="6302dec0319e422a0e1e8fd6b281574e82d4a3a90d6bd87dcbe6bae277cd68fd" exitCode=0
Oct 31 07:38:03.981295 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:03.981287    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" event={"ID":"023261ef-dadc-4c2e-ad06-5d4c1eba1025","Type":"ContainerDied","Data":"6302dec0319e422a0e1e8fd6b281574e82d4a3a90d6bd87dcbe6bae277cd68fd"}
Oct 31 07:38:03.983008 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:03.982987    4322 generic.go:356] "Generic (PLEG): container finished" podID="efa5e541-06f0-4ece-a948-5e3037e225de" containerID="86faed514e5d875234d462ac1dd123821f99d5916d34763e589837c002084769" exitCode=0
Oct 31 07:38:03.983134 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:03.983015    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" event={"ID":"efa5e541-06f0-4ece-a948-5e3037e225de","Type":"ContainerDied","Data":"86faed514e5d875234d462ac1dd123821f99d5916d34763e589837c002084769"}
Oct 31 07:38:05.004875 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:05.004838    4322 generic.go:356] "Generic (PLEG): container finished" podID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerID="0b855d77eec48c3fc9963475760e651a6bd1ef3f8da5fb0b0992e6459191119a" exitCode=0
Oct 31 07:38:05.005539 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:05.004941    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" event={"ID":"f0c8980e-1f7e-480b-a633-e7317a35b7e4","Type":"ContainerDied","Data":"0b855d77eec48c3fc9963475760e651a6bd1ef3f8da5fb0b0992e6459191119a"}
Oct 31 07:38:31.720055 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:38:31.720016    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:38:33.276429 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:33.276390    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" event={"ID":"023261ef-dadc-4c2e-ad06-5d4c1eba1025","Type":"ContainerStarted","Data":"d8c49aa026598dc170de70c8c890f3d733ed75a57af5bd509abb6ee0d8463af6"}
Oct 31 07:38:33.279703 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:33.279674    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" event={"ID":"f0c8980e-1f7e-480b-a633-e7317a35b7e4","Type":"ContainerStarted","Data":"01e000d020e94d579be9ba7281fb96cd6a2a1a2263007f62e58b7619166faf42"}
Oct 31 07:38:33.282834 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:33.282753    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" event={"ID":"efa5e541-06f0-4ece-a948-5e3037e225de","Type":"ContainerStarted","Data":"ebfbada157b11968e2345dfc9264e2d9a8eaf84840727973e6502fc113a11cde"}
Oct 31 07:38:36.304691 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:36.304656    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" event={"ID":"f0c8980e-1f7e-480b-a633-e7317a35b7e4","Type":"ContainerStarted","Data":"0dc698f062ea50f2bc3ee3d520017e1ac2129ffcbc7e74cfde43fdc6157c8480"}
Oct 31 07:38:36.306743 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:36.306713    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" event={"ID":"efa5e541-06f0-4ece-a948-5e3037e225de","Type":"ContainerStarted","Data":"708828652aca06045e6640a1c137226ed7c07f135cad1e5bbb6cf5c49eac8ef0"}
Oct 31 07:38:36.306885 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:36.306777    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:38:36.306885 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:36.306813    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:38:36.308595 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:36.308564    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 31 07:38:36.308904 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:36.308885    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" event={"ID":"023261ef-dadc-4c2e-ad06-5d4c1eba1025","Type":"ContainerStarted","Data":"d9cf47951b3e578ba9f3495bcfed23fe85f12de964071ea7b6891645153425d3"}
Oct 31 07:38:36.309046 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:36.309032    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:38:36.309123 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:36.309083    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:38:36.310063 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:36.310043    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Oct 31 07:38:36.364817 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:36.364757    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" podStartSLOduration=2.68945077 podStartE2EDuration="38.364738973s" totalImagesPullingTime="31.720235183s" totalInitContainerRuntime="3.95505302s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:37:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:38:36.362527924 +0000 UTC m=+557.626348534" watchObservedRunningTime="2026-10-31 07:38:36.364738973 +0000 UTC m=+557.628559586"
Oct 31 07:38:36.406549 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:36.406504    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" podStartSLOduration=2.368222374 podStartE2EDuration="38.406491511s" totalImagesPullingTime="31.938803663s" totalInitContainerRuntime="4.099465474s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:37:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:38:36.404893515 +0000 UTC m=+557.668714129" watchObservedRunningTime="2026-10-31 07:38:36.406491511 +0000 UTC m=+557.670312122"
Oct 31 07:38:37.314032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:37.313981    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Oct 31 07:38:37.314746 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:37.314713    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 31 07:38:39.324713 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:39.324675    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" event={"ID":"f0c8980e-1f7e-480b-a633-e7317a35b7e4","Type":"ContainerStarted","Data":"dbb180cb4c5067a74ac32ccb2afed7bcb5d0d4e87d280f10c224723f84489758"}
Oct 31 07:38:39.325201 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:39.324818    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:38:39.393754 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:39.393704    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podStartSLOduration=3.367833006 podStartE2EDuration="40.393690065s" totalImagesPullingTime="33.106832907s" totalInitContainerRuntime="3.919024152s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:37:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:38:39.380950913 +0000 UTC m=+560.644771524" watchObservedRunningTime="2026-10-31 07:38:39.393690065 +0000 UTC m=+560.657510676"
Oct 31 07:38:40.329431 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:40.329400    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:38:40.329904 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:40.329462    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:38:40.330948 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:40.330919    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 31 07:38:40.331585 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:40.331561    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:38:41.333480 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:41.333438    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 31 07:38:41.333947 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:41.333903    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:38:41.337398 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:41.337380    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:38:42.319015 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:42.318984    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:38:42.319314 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:42.319295    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:38:42.319523 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:42.319500    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Oct 31 07:38:42.319906 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:42.319882    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 31 07:38:42.337347 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:42.337321    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 31 07:38:42.337724 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:42.337581    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:38:52.319705 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:52.319670    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Oct 31 07:38:52.336183 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:52.319956    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 31 07:38:52.337881 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:52.337853    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 31 07:38:52.338406 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:38:52.338381    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:39:01.709925 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:39:01.709844    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:39:02.319525 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:02.319482    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Oct 31 07:39:02.319945 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:02.319915    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 31 07:39:02.337411 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:02.337368    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 31 07:39:02.337765 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:02.337739    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:39:12.320211 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:12.320173    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 31 07:39:12.320677 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:12.320174    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Oct 31 07:39:12.338059 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:12.338035    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 31 07:39:12.338506 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:12.338483    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:39:22.319745 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:22.319703    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Oct 31 07:39:22.320263 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:22.319827    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 31 07:39:22.337307 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:22.337274    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 31 07:39:22.337707 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:22.337684    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:39:31.711065 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:39:31.711020    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:39:32.319739 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:32.319702    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Oct 31 07:39:32.319935 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:32.319839    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 31 07:39:32.337239 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:32.337200    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 31 07:39:32.337650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:32.337631    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:39:42.319768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:42.319723    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Oct 31 07:39:42.320198 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:42.319844    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 31 07:39:42.337459 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:42.337415    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 31 07:39:42.337824 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:42.337786    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:39:47.865933 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:47.865886    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 31 07:39:49.865847 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:49.865805    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Oct 31 07:39:52.337985 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:52.337936    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 31 07:39:52.338477 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:52.338417    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:39:57.866392 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:57.866364    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:39:58.331717 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:58.331683    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"]
Oct 31 07:39:58.331929 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:58.331902    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kserve-container" containerID="cri-o://d8c49aa026598dc170de70c8c890f3d733ed75a57af5bd509abb6ee0d8463af6" gracePeriod=30
Oct 31 07:39:58.332049 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:58.331990    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kube-rbac-proxy" containerID="cri-o://d9cf47951b3e578ba9f3495bcfed23fe85f12de964071ea7b6891645153425d3" gracePeriod=30
Oct 31 07:39:58.333007 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:58.332988    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:39:58.683821 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:58.683747    4322 generic.go:356] "Generic (PLEG): container finished" podID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerID="d9cf47951b3e578ba9f3495bcfed23fe85f12de964071ea7b6891645153425d3" exitCode=0
Oct 31 07:39:58.683821 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:39:58.683780    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" event={"ID":"023261ef-dadc-4c2e-ad06-5d4c1eba1025","Type":"ContainerDied","Data":"d9cf47951b3e578ba9f3495bcfed23fe85f12de964071ea7b6891645153425d3"}
Oct 31 07:40:01.711350 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:40:01.711303    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:40:02.314423 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:02.314378    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Oct 31 07:40:02.338467 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:02.338436    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:40:02.338634 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:02.338559    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:40:03.280032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.280008    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:40:03.371622 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.371559    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/023261ef-dadc-4c2e-ad06-5d4c1eba1025-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/023261ef-dadc-4c2e-ad06-5d4c1eba1025-cabundle-cert\") pod \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") "
Oct 31 07:40:03.371742 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.371656    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kube-rbac-proxy-sar-config\") pod \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") "
Oct 31 07:40:03.371742 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.371718    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kube-api-access-dxqsv\" (UniqueName: \"kubernetes.io/projected/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kube-api-access-dxqsv\") pod \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") "
Oct 31 07:40:03.371821 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.371750    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/023261ef-dadc-4c2e-ad06-5d4c1eba1025-proxy-tls\" (UniqueName: \"kubernetes.io/secret/023261ef-dadc-4c2e-ad06-5d4c1eba1025-proxy-tls\") pod \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") "
Oct 31 07:40:03.371821 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.371790    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kserve-provision-location\") pod \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\" (UID: \"023261ef-dadc-4c2e-ad06-5d4c1eba1025\") "
Oct 31 07:40:03.371928 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.371899    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/023261ef-dadc-4c2e-ad06-5d4c1eba1025-cabundle-cert" pod "023261ef-dadc-4c2e-ad06-5d4c1eba1025" (UID: "023261ef-dadc-4c2e-ad06-5d4c1eba1025"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:40:03.372132 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.372109    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kube-rbac-proxy-sar-config" pod "023261ef-dadc-4c2e-ad06-5d4c1eba1025" (UID: "023261ef-dadc-4c2e-ad06-5d4c1eba1025"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:40:03.372288 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.372153    4322 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/023261ef-dadc-4c2e-ad06-5d4c1eba1025-cabundle-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:40:03.372288 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.372168    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kserve-provision-location" pod "023261ef-dadc-4c2e-ad06-5d4c1eba1025" (UID: "023261ef-dadc-4c2e-ad06-5d4c1eba1025"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:40:03.373889 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.373866    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/023261ef-dadc-4c2e-ad06-5d4c1eba1025-proxy-tls" pod "023261ef-dadc-4c2e-ad06-5d4c1eba1025" (UID: "023261ef-dadc-4c2e-ad06-5d4c1eba1025"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:40:03.373979 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.373964    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kube-api-access-dxqsv" pod "023261ef-dadc-4c2e-ad06-5d4c1eba1025" (UID: "023261ef-dadc-4c2e-ad06-5d4c1eba1025"). InnerVolumeSpecName "kube-api-access-dxqsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:40:03.472646 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.472619    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:40:03.472646 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.472642    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxqsv\" (UniqueName: \"kubernetes.io/projected/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kube-api-access-dxqsv\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:40:03.472787 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.472657    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/023261ef-dadc-4c2e-ad06-5d4c1eba1025-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:40:03.472787 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.472671    4322 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/023261ef-dadc-4c2e-ad06-5d4c1eba1025-kserve-provision-location\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:40:03.708393 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.708315    4322 generic.go:356] "Generic (PLEG): container finished" podID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerID="d8c49aa026598dc170de70c8c890f3d733ed75a57af5bd509abb6ee0d8463af6" exitCode=0
Oct 31 07:40:03.708557 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.708406    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" event={"ID":"023261ef-dadc-4c2e-ad06-5d4c1eba1025","Type":"ContainerDied","Data":"d8c49aa026598dc170de70c8c890f3d733ed75a57af5bd509abb6ee0d8463af6"}
Oct 31 07:40:03.708557 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.708461    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp" event={"ID":"023261ef-dadc-4c2e-ad06-5d4c1eba1025","Type":"ContainerDied","Data":"a3e79e587362ea2ef86fbed15acd614c456ea896b144879ca506fdbed572a43f"}
Oct 31 07:40:03.708557 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.708510    4322 scope.go:122] "RemoveContainer" containerID="d9cf47951b3e578ba9f3495bcfed23fe85f12de964071ea7b6891645153425d3"
Oct 31 07:40:03.708723 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.708677    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"
Oct 31 07:40:03.721124 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.721103    4322 scope.go:122] "RemoveContainer" containerID="d8c49aa026598dc170de70c8c890f3d733ed75a57af5bd509abb6ee0d8463af6"
Oct 31 07:40:03.728444 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.728424    4322 scope.go:122] "RemoveContainer" containerID="6302dec0319e422a0e1e8fd6b281574e82d4a3a90d6bd87dcbe6bae277cd68fd"
Oct 31 07:40:03.735620 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.735591    4322 scope.go:122] "RemoveContainer" containerID="d9cf47951b3e578ba9f3495bcfed23fe85f12de964071ea7b6891645153425d3"
Oct 31 07:40:03.735868 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:40:03.735848    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9cf47951b3e578ba9f3495bcfed23fe85f12de964071ea7b6891645153425d3\": container with ID starting with d9cf47951b3e578ba9f3495bcfed23fe85f12de964071ea7b6891645153425d3 not found: ID does not exist" containerID="d9cf47951b3e578ba9f3495bcfed23fe85f12de964071ea7b6891645153425d3"
Oct 31 07:40:03.735940 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.735876    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9cf47951b3e578ba9f3495bcfed23fe85f12de964071ea7b6891645153425d3"} err="failed to get container status \"d9cf47951b3e578ba9f3495bcfed23fe85f12de964071ea7b6891645153425d3\": rpc error: code = NotFound desc = could not find container \"d9cf47951b3e578ba9f3495bcfed23fe85f12de964071ea7b6891645153425d3\": container with ID starting with d9cf47951b3e578ba9f3495bcfed23fe85f12de964071ea7b6891645153425d3 not found: ID does not exist"
Oct 31 07:40:03.735940 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.735894    4322 scope.go:122] "RemoveContainer" containerID="d8c49aa026598dc170de70c8c890f3d733ed75a57af5bd509abb6ee0d8463af6"
Oct 31 07:40:03.736152 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:40:03.736135    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8c49aa026598dc170de70c8c890f3d733ed75a57af5bd509abb6ee0d8463af6\": container with ID starting with d8c49aa026598dc170de70c8c890f3d733ed75a57af5bd509abb6ee0d8463af6 not found: ID does not exist" containerID="d8c49aa026598dc170de70c8c890f3d733ed75a57af5bd509abb6ee0d8463af6"
Oct 31 07:40:03.736201 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.736166    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8c49aa026598dc170de70c8c890f3d733ed75a57af5bd509abb6ee0d8463af6"} err="failed to get container status \"d8c49aa026598dc170de70c8c890f3d733ed75a57af5bd509abb6ee0d8463af6\": rpc error: code = NotFound desc = could not find container \"d8c49aa026598dc170de70c8c890f3d733ed75a57af5bd509abb6ee0d8463af6\": container with ID starting with d8c49aa026598dc170de70c8c890f3d733ed75a57af5bd509abb6ee0d8463af6 not found: ID does not exist"
Oct 31 07:40:03.736201 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.736193    4322 scope.go:122] "RemoveContainer" containerID="6302dec0319e422a0e1e8fd6b281574e82d4a3a90d6bd87dcbe6bae277cd68fd"
Oct 31 07:40:03.736419 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:40:03.736404    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6302dec0319e422a0e1e8fd6b281574e82d4a3a90d6bd87dcbe6bae277cd68fd\": container with ID starting with 6302dec0319e422a0e1e8fd6b281574e82d4a3a90d6bd87dcbe6bae277cd68fd not found: ID does not exist" containerID="6302dec0319e422a0e1e8fd6b281574e82d4a3a90d6bd87dcbe6bae277cd68fd"
Oct 31 07:40:03.736482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.736422    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6302dec0319e422a0e1e8fd6b281574e82d4a3a90d6bd87dcbe6bae277cd68fd"} err="failed to get container status \"6302dec0319e422a0e1e8fd6b281574e82d4a3a90d6bd87dcbe6bae277cd68fd\": rpc error: code = NotFound desc = could not find container \"6302dec0319e422a0e1e8fd6b281574e82d4a3a90d6bd87dcbe6bae277cd68fd\": container with ID starting with 6302dec0319e422a0e1e8fd6b281574e82d4a3a90d6bd87dcbe6bae277cd68fd not found: ID does not exist"
Oct 31 07:40:03.762542 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.762518    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"]
Oct 31 07:40:03.774630 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:03.774609    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-wpkkp"]
Oct 31 07:40:04.871176 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:04.871143    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" path="/var/lib/kubelet/pods/023261ef-dadc-4c2e-ad06-5d4c1eba1025/volumes"
Oct 31 07:40:08.945174 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:08.945142    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"]
Oct 31 07:40:08.945633 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:08.945364    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kserve-container" containerID="cri-o://ebfbada157b11968e2345dfc9264e2d9a8eaf84840727973e6502fc113a11cde" gracePeriod=30
Oct 31 07:40:08.945633 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:08.945417    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kube-rbac-proxy" containerID="cri-o://708828652aca06045e6640a1c137226ed7c07f135cad1e5bbb6cf5c49eac8ef0" gracePeriod=30
Oct 31 07:40:09.135599 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.135572    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"]
Oct 31 07:40:09.135962 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.135948    4322 cpu_manager.go:414] "containerMap: removing container" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kserve-container"
Oct 31 07:40:09.136008 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.135963    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kserve-container"
Oct 31 07:40:09.136008 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.135979    4322 cpu_manager.go:414] "containerMap: removing container" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="storage-initializer"
Oct 31 07:40:09.136008 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.135984    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="storage-initializer"
Oct 31 07:40:09.136008 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.136001    4322 cpu_manager.go:414] "containerMap: removing container" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kube-rbac-proxy"
Oct 31 07:40:09.136008 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.136007    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kube-rbac-proxy"
Oct 31 07:40:09.136165 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.136101    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kube-rbac-proxy"
Oct 31 07:40:09.136165 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.136121    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="023261ef-dadc-4c2e-ad06-5d4c1eba1025" containerName="kserve-container"
Oct 31 07:40:09.139559 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.139536    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.154013 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.153989    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7281997251\"/\"isvc-canary-rollback-predictor-serving-cert\""
Oct 31 07:40:09.160642 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.160619    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"]
Oct 31 07:40:09.222789 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.222734    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c19c27a-86bb-448b-bc1b-59d001ca9292-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-5b7b487756-6tdtw\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.222789 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.222775    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c19c27a-86bb-448b-bc1b-59d001ca9292-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-5b7b487756-6tdtw\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.222933 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.222864    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8c19c27a-86bb-448b-bc1b-59d001ca9292-cabundle-cert\") pod \"isvc-canary-rollback-predictor-5b7b487756-6tdtw\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.222933 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.222891    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c19c27a-86bb-448b-bc1b-59d001ca9292-proxy-tls\") pod \"isvc-canary-rollback-predictor-5b7b487756-6tdtw\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.222933 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.222911    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jbmp\" (UniqueName: \"kubernetes.io/projected/8c19c27a-86bb-448b-bc1b-59d001ca9292-kube-api-access-4jbmp\") pod \"isvc-canary-rollback-predictor-5b7b487756-6tdtw\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.324380 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.324350    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c19c27a-86bb-448b-bc1b-59d001ca9292-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-5b7b487756-6tdtw\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.324517 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.324396    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c19c27a-86bb-448b-bc1b-59d001ca9292-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-5b7b487756-6tdtw\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.324517 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.324451    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8c19c27a-86bb-448b-bc1b-59d001ca9292-cabundle-cert\") pod \"isvc-canary-rollback-predictor-5b7b487756-6tdtw\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.324517 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.324482    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c19c27a-86bb-448b-bc1b-59d001ca9292-proxy-tls\") pod \"isvc-canary-rollback-predictor-5b7b487756-6tdtw\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.324517 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.324505    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jbmp\" (UniqueName: \"kubernetes.io/projected/8c19c27a-86bb-448b-bc1b-59d001ca9292-kube-api-access-4jbmp\") pod \"isvc-canary-rollback-predictor-5b7b487756-6tdtw\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.324945 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.324919    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c19c27a-86bb-448b-bc1b-59d001ca9292-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-5b7b487756-6tdtw\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.325116 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.325093    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c19c27a-86bb-448b-bc1b-59d001ca9292-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-5b7b487756-6tdtw\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.325116 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.325104    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8c19c27a-86bb-448b-bc1b-59d001ca9292-cabundle-cert\") pod \"isvc-canary-rollback-predictor-5b7b487756-6tdtw\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.326898 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.326879    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c19c27a-86bb-448b-bc1b-59d001ca9292-proxy-tls\") pod \"isvc-canary-rollback-predictor-5b7b487756-6tdtw\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.347075 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.347055    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jbmp\" (UniqueName: \"kubernetes.io/projected/8c19c27a-86bb-448b-bc1b-59d001ca9292-kube-api-access-4jbmp\") pod \"isvc-canary-rollback-predictor-5b7b487756-6tdtw\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") " pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.456449 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.456418    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:09.736622 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.736586    4322 generic.go:356] "Generic (PLEG): container finished" podID="efa5e541-06f0-4ece-a948-5e3037e225de" containerID="708828652aca06045e6640a1c137226ed7c07f135cad1e5bbb6cf5c49eac8ef0" exitCode=0
Oct 31 07:40:09.736783 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.736658    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" event={"ID":"efa5e541-06f0-4ece-a948-5e3037e225de","Type":"ContainerDied","Data":"708828652aca06045e6640a1c137226ed7c07f135cad1e5bbb6cf5c49eac8ef0"}
Oct 31 07:40:09.758699 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:09.758678    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"]
Oct 31 07:40:09.761539 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:40:09.761516    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c19c27a_86bb_448b_bc1b_59d001ca9292.slice/crio-806efa4aff6573e147d7d4dd0dfd0793c6269c4e4883dd941b4141bfab9a0191 WatchSource:0}: Error finding container 806efa4aff6573e147d7d4dd0dfd0793c6269c4e4883dd941b4141bfab9a0191: Status 404 returned error can't find the container with id 806efa4aff6573e147d7d4dd0dfd0793c6269c4e4883dd941b4141bfab9a0191
Oct 31 07:40:10.743015 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:10.742969    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" event={"ID":"8c19c27a-86bb-448b-bc1b-59d001ca9292","Type":"ContainerStarted","Data":"cc082cead0978ce4f63de13c4648df99f5803b82352ede85844febe49988a6a7"}
Oct 31 07:40:10.743507 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:10.743079    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" event={"ID":"8c19c27a-86bb-448b-bc1b-59d001ca9292","Type":"ContainerStarted","Data":"806efa4aff6573e147d7d4dd0dfd0793c6269c4e4883dd941b4141bfab9a0191"}
Oct 31 07:40:12.314755 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:12.314714    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Oct 31 07:40:13.693547 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.693492    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:40:13.761805 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.761773    4322 generic.go:356] "Generic (PLEG): container finished" podID="efa5e541-06f0-4ece-a948-5e3037e225de" containerID="ebfbada157b11968e2345dfc9264e2d9a8eaf84840727973e6502fc113a11cde" exitCode=0
Oct 31 07:40:13.761964 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.761832    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"
Oct 31 07:40:13.761964 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.761833    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" event={"ID":"efa5e541-06f0-4ece-a948-5e3037e225de","Type":"ContainerDied","Data":"ebfbada157b11968e2345dfc9264e2d9a8eaf84840727973e6502fc113a11cde"}
Oct 31 07:40:13.761964 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.761896    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b" event={"ID":"efa5e541-06f0-4ece-a948-5e3037e225de","Type":"ContainerDied","Data":"c79b2d15bf1937939e38c63df22f752c7d95cd94b50354a916cf5cb12e2780eb"}
Oct 31 07:40:13.761964 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.761936    4322 scope.go:122] "RemoveContainer" containerID="708828652aca06045e6640a1c137226ed7c07f135cad1e5bbb6cf5c49eac8ef0"
Oct 31 07:40:13.763550 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.763528    4322 generic.go:356] "Generic (PLEG): container finished" podID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerID="cc082cead0978ce4f63de13c4648df99f5803b82352ede85844febe49988a6a7" exitCode=0
Oct 31 07:40:13.763666 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.763556    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" event={"ID":"8c19c27a-86bb-448b-bc1b-59d001ca9292","Type":"ContainerDied","Data":"cc082cead0978ce4f63de13c4648df99f5803b82352ede85844febe49988a6a7"}
Oct 31 07:40:13.764772 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.764753    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/efa5e541-06f0-4ece-a948-5e3037e225de-proxy-tls\" (UniqueName: \"kubernetes.io/secret/efa5e541-06f0-4ece-a948-5e3037e225de-proxy-tls\") pod \"efa5e541-06f0-4ece-a948-5e3037e225de\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") "
Oct 31 07:40:13.764877 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.764801    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efa5e541-06f0-4ece-a948-5e3037e225de-kube-api-access-rlscf\" (UniqueName: \"kubernetes.io/projected/efa5e541-06f0-4ece-a948-5e3037e225de-kube-api-access-rlscf\") pod \"efa5e541-06f0-4ece-a948-5e3037e225de\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") "
Oct 31 07:40:13.764877 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.764831    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efa5e541-06f0-4ece-a948-5e3037e225de-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/efa5e541-06f0-4ece-a948-5e3037e225de-kserve-provision-location\") pod \"efa5e541-06f0-4ece-a948-5e3037e225de\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") "
Oct 31 07:40:13.764877 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.764866    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/efa5e541-06f0-4ece-a948-5e3037e225de-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/efa5e541-06f0-4ece-a948-5e3037e225de-kube-rbac-proxy-sar-config\") pod \"efa5e541-06f0-4ece-a948-5e3037e225de\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") "
Oct 31 07:40:13.765039 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.764937    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/efa5e541-06f0-4ece-a948-5e3037e225de-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/efa5e541-06f0-4ece-a948-5e3037e225de-cabundle-cert\") pod \"efa5e541-06f0-4ece-a948-5e3037e225de\" (UID: \"efa5e541-06f0-4ece-a948-5e3037e225de\") "
Oct 31 07:40:13.765401 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.765359    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa5e541-06f0-4ece-a948-5e3037e225de-kserve-provision-location" pod "efa5e541-06f0-4ece-a948-5e3037e225de" (UID: "efa5e541-06f0-4ece-a948-5e3037e225de"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:40:13.766337 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.766310    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa5e541-06f0-4ece-a948-5e3037e225de-cabundle-cert" pod "efa5e541-06f0-4ece-a948-5e3037e225de" (UID: "efa5e541-06f0-4ece-a948-5e3037e225de"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:40:13.766459 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.766438    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa5e541-06f0-4ece-a948-5e3037e225de-kube-rbac-proxy-sar-config" pod "efa5e541-06f0-4ece-a948-5e3037e225de" (UID: "efa5e541-06f0-4ece-a948-5e3037e225de"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:40:13.767405 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.767378    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efa5e541-06f0-4ece-a948-5e3037e225de-proxy-tls" pod "efa5e541-06f0-4ece-a948-5e3037e225de" (UID: "efa5e541-06f0-4ece-a948-5e3037e225de"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:40:13.773192 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.773176    4322 scope.go:122] "RemoveContainer" containerID="ebfbada157b11968e2345dfc9264e2d9a8eaf84840727973e6502fc113a11cde"
Oct 31 07:40:13.774349 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.774324    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa5e541-06f0-4ece-a948-5e3037e225de-kube-api-access-rlscf" pod "efa5e541-06f0-4ece-a948-5e3037e225de" (UID: "efa5e541-06f0-4ece-a948-5e3037e225de"). InnerVolumeSpecName "kube-api-access-rlscf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:40:13.782399 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.782365    4322 scope.go:122] "RemoveContainer" containerID="86faed514e5d875234d462ac1dd123821f99d5916d34763e589837c002084769"
Oct 31 07:40:13.794823 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.794802    4322 scope.go:122] "RemoveContainer" containerID="708828652aca06045e6640a1c137226ed7c07f135cad1e5bbb6cf5c49eac8ef0"
Oct 31 07:40:13.795124 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:40:13.795107    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"708828652aca06045e6640a1c137226ed7c07f135cad1e5bbb6cf5c49eac8ef0\": container with ID starting with 708828652aca06045e6640a1c137226ed7c07f135cad1e5bbb6cf5c49eac8ef0 not found: ID does not exist" containerID="708828652aca06045e6640a1c137226ed7c07f135cad1e5bbb6cf5c49eac8ef0"
Oct 31 07:40:13.795170 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.795133    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"708828652aca06045e6640a1c137226ed7c07f135cad1e5bbb6cf5c49eac8ef0"} err="failed to get container status \"708828652aca06045e6640a1c137226ed7c07f135cad1e5bbb6cf5c49eac8ef0\": rpc error: code = NotFound desc = could not find container \"708828652aca06045e6640a1c137226ed7c07f135cad1e5bbb6cf5c49eac8ef0\": container with ID starting with 708828652aca06045e6640a1c137226ed7c07f135cad1e5bbb6cf5c49eac8ef0 not found: ID does not exist"
Oct 31 07:40:13.795170 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.795148    4322 scope.go:122] "RemoveContainer" containerID="ebfbada157b11968e2345dfc9264e2d9a8eaf84840727973e6502fc113a11cde"
Oct 31 07:40:13.795404 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:40:13.795386    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebfbada157b11968e2345dfc9264e2d9a8eaf84840727973e6502fc113a11cde\": container with ID starting with ebfbada157b11968e2345dfc9264e2d9a8eaf84840727973e6502fc113a11cde not found: ID does not exist" containerID="ebfbada157b11968e2345dfc9264e2d9a8eaf84840727973e6502fc113a11cde"
Oct 31 07:40:13.795459 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.795409    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebfbada157b11968e2345dfc9264e2d9a8eaf84840727973e6502fc113a11cde"} err="failed to get container status \"ebfbada157b11968e2345dfc9264e2d9a8eaf84840727973e6502fc113a11cde\": rpc error: code = NotFound desc = could not find container \"ebfbada157b11968e2345dfc9264e2d9a8eaf84840727973e6502fc113a11cde\": container with ID starting with ebfbada157b11968e2345dfc9264e2d9a8eaf84840727973e6502fc113a11cde not found: ID does not exist"
Oct 31 07:40:13.795459 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.795422    4322 scope.go:122] "RemoveContainer" containerID="86faed514e5d875234d462ac1dd123821f99d5916d34763e589837c002084769"
Oct 31 07:40:13.795950 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:40:13.795937    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86faed514e5d875234d462ac1dd123821f99d5916d34763e589837c002084769\": container with ID starting with 86faed514e5d875234d462ac1dd123821f99d5916d34763e589837c002084769 not found: ID does not exist" containerID="86faed514e5d875234d462ac1dd123821f99d5916d34763e589837c002084769"
Oct 31 07:40:13.795998 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.795954    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86faed514e5d875234d462ac1dd123821f99d5916d34763e589837c002084769"} err="failed to get container status \"86faed514e5d875234d462ac1dd123821f99d5916d34763e589837c002084769\": rpc error: code = NotFound desc = could not find container \"86faed514e5d875234d462ac1dd123821f99d5916d34763e589837c002084769\": container with ID starting with 86faed514e5d875234d462ac1dd123821f99d5916d34763e589837c002084769 not found: ID does not exist"
Oct 31 07:40:13.866839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.866661    4322 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/efa5e541-06f0-4ece-a948-5e3037e225de-cabundle-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:40:13.866839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.866689    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/efa5e541-06f0-4ece-a948-5e3037e225de-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:40:13.866839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.866703    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlscf\" (UniqueName: \"kubernetes.io/projected/efa5e541-06f0-4ece-a948-5e3037e225de-kube-api-access-rlscf\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:40:13.866839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.866716    4322 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/efa5e541-06f0-4ece-a948-5e3037e225de-kserve-provision-location\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:40:13.866839 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:13.866730    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/efa5e541-06f0-4ece-a948-5e3037e225de-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:40:14.106960 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:14.106934    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"]
Oct 31 07:40:14.110755 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:14.110734    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-v2-predictor-588d4d6495-x7x6b"]
Oct 31 07:40:14.486990 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:14.486912    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"]
Oct 31 07:40:14.487333 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:14.487284    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container" containerID="cri-o://01e000d020e94d579be9ba7281fb96cd6a2a1a2263007f62e58b7619166faf42" gracePeriod=30
Oct 31 07:40:14.487686 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:14.487645    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent" containerID="cri-o://dbb180cb4c5067a74ac32ccb2afed7bcb5d0d4e87d280f10c224723f84489758" gracePeriod=30
Oct 31 07:40:14.487859 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:14.487840    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kube-rbac-proxy" containerID="cri-o://0dc698f062ea50f2bc3ee3d520017e1ac2129ffcbc7e74cfde43fdc6157c8480" gracePeriod=30
Oct 31 07:40:14.771165 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:14.771138    4322 generic.go:356] "Generic (PLEG): container finished" podID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerID="0dc698f062ea50f2bc3ee3d520017e1ac2129ffcbc7e74cfde43fdc6157c8480" exitCode=0
Oct 31 07:40:14.771570 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:14.771212    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" event={"ID":"f0c8980e-1f7e-480b-a633-e7317a35b7e4","Type":"ContainerDied","Data":"0dc698f062ea50f2bc3ee3d520017e1ac2129ffcbc7e74cfde43fdc6157c8480"}
Oct 31 07:40:14.774268 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:14.774247    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" event={"ID":"8c19c27a-86bb-448b-bc1b-59d001ca9292","Type":"ContainerStarted","Data":"663a00148e8cf1b8b7995e64fd510b3dd88770da872f43796138bfd6eac391c8"}
Oct 31 07:40:14.774392 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:14.774299    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" event={"ID":"8c19c27a-86bb-448b-bc1b-59d001ca9292","Type":"ContainerStarted","Data":"59bdc692835ce52a4b308780b36a2c0964927dac667f353519ecae51587ff24e"}
Oct 31 07:40:14.774498 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:14.774484    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:14.819558 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:14.819511    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" podStartSLOduration=2.202756988 podStartE2EDuration="5.819498802s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.616741814s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:40:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:40:14.817952751 +0000 UTC m=+656.081773361" watchObservedRunningTime="2026-10-31 07:40:14.819498802 +0000 UTC m=+656.083319412"
Oct 31 07:40:14.871691 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:14.871662    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" path="/var/lib/kubelet/pods/efa5e541-06f0-4ece-a948-5e3037e225de/volumes"
Oct 31 07:40:15.778917 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:15.778885    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:15.780113 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:15.780085    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 31 07:40:16.334581 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:16.334539    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Oct 31 07:40:16.782742 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:16.782701    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 31 07:40:19.798474 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:19.798443    4322 generic.go:356] "Generic (PLEG): container finished" podID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerID="01e000d020e94d579be9ba7281fb96cd6a2a1a2263007f62e58b7619166faf42" exitCode=0
Oct 31 07:40:19.798865 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:19.798518    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" event={"ID":"f0c8980e-1f7e-480b-a633-e7317a35b7e4","Type":"ContainerDied","Data":"01e000d020e94d579be9ba7281fb96cd6a2a1a2263007f62e58b7619166faf42"}
Oct 31 07:40:21.333873 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:21.333833    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Oct 31 07:40:21.787536 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:21.787509    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:40:21.788069 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:21.788042    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 31 07:40:22.338024 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:22.337978    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 31 07:40:22.338405 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:22.338307    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:40:26.333761 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:26.333722    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Oct 31 07:40:26.334217 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:26.333854    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:40:31.334501 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:31.334420    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Oct 31 07:40:31.710418 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:40:31.710336    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:40:31.788131 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:31.788093    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 31 07:40:32.337821 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:32.337777    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 31 07:40:32.338282 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:32.338143    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:40:36.334073 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:36.334026    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Oct 31 07:40:41.333978 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:41.333934    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Oct 31 07:40:41.788970 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:41.788931    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 31 07:40:42.337538 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:42.337493    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 31 07:40:42.337969 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:42.337682    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:40:42.337969 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:42.337829    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:40:42.337969 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:42.337928    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:40:44.632999 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.632973    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:40:44.719777 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.719740    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kube-api-access-rxp6g\" (UniqueName: \"kubernetes.io/projected/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kube-api-access-rxp6g\") pod \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") "
Oct 31 07:40:44.719962 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.719811    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0c8980e-1f7e-480b-a633-e7317a35b7e4-proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0c8980e-1f7e-480b-a633-e7317a35b7e4-proxy-tls\") pod \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") "
Oct 31 07:40:44.719962 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.719911    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kserve-provision-location\") pod \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") "
Oct 31 07:40:44.719962 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.719944    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0c8980e-1f7e-480b-a633-e7317a35b7e4-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f0c8980e-1f7e-480b-a633-e7317a35b7e4-cabundle-cert\") pod \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") "
Oct 31 07:40:44.720148 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.719989    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kube-rbac-proxy-sar-config\") pod \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\" (UID: \"f0c8980e-1f7e-480b-a633-e7317a35b7e4\") "
Oct 31 07:40:44.720276 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.720248    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kserve-provision-location" pod "f0c8980e-1f7e-480b-a633-e7317a35b7e4" (UID: "f0c8980e-1f7e-480b-a633-e7317a35b7e4"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:40:44.720368 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.720314    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0c8980e-1f7e-480b-a633-e7317a35b7e4-cabundle-cert" pod "f0c8980e-1f7e-480b-a633-e7317a35b7e4" (UID: "f0c8980e-1f7e-480b-a633-e7317a35b7e4"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:40:44.720523 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.720499    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kube-rbac-proxy-sar-config" pod "f0c8980e-1f7e-480b-a633-e7317a35b7e4" (UID: "f0c8980e-1f7e-480b-a633-e7317a35b7e4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:40:44.722107 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.722086    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kube-api-access-rxp6g" pod "f0c8980e-1f7e-480b-a633-e7317a35b7e4" (UID: "f0c8980e-1f7e-480b-a633-e7317a35b7e4"). InnerVolumeSpecName "kube-api-access-rxp6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:40:44.722216 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.722117    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0c8980e-1f7e-480b-a633-e7317a35b7e4-proxy-tls" pod "f0c8980e-1f7e-480b-a633-e7317a35b7e4" (UID: "f0c8980e-1f7e-480b-a633-e7317a35b7e4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:40:44.820988 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.820961    4322 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kserve-provision-location\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:40:44.820988 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.820987    4322 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f0c8980e-1f7e-480b-a633-e7317a35b7e4-cabundle-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:40:44.821160 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.820997    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:40:44.821160 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.821007    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxp6g\" (UniqueName: \"kubernetes.io/projected/f0c8980e-1f7e-480b-a633-e7317a35b7e4-kube-api-access-rxp6g\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:40:44.821160 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.821017    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0c8980e-1f7e-480b-a633-e7317a35b7e4-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:40:44.910501 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.910473    4322 generic.go:356] "Generic (PLEG): container finished" podID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerID="dbb180cb4c5067a74ac32ccb2afed7bcb5d0d4e87d280f10c224723f84489758" exitCode=0
Oct 31 07:40:44.910641 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.910539    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"
Oct 31 07:40:44.910641 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.910547    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" event={"ID":"f0c8980e-1f7e-480b-a633-e7317a35b7e4","Type":"ContainerDied","Data":"dbb180cb4c5067a74ac32ccb2afed7bcb5d0d4e87d280f10c224723f84489758"}
Oct 31 07:40:44.910641 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.910596    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74" event={"ID":"f0c8980e-1f7e-480b-a633-e7317a35b7e4","Type":"ContainerDied","Data":"0528df177c888156067e976babd8905b9be528fda02cd92b00544ab0c7f05cb2"}
Oct 31 07:40:44.910641 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.910627    4322 scope.go:122] "RemoveContainer" containerID="dbb180cb4c5067a74ac32ccb2afed7bcb5d0d4e87d280f10c224723f84489758"
Oct 31 07:40:44.918698 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.918681    4322 scope.go:122] "RemoveContainer" containerID="0dc698f062ea50f2bc3ee3d520017e1ac2129ffcbc7e74cfde43fdc6157c8480"
Oct 31 07:40:44.926052 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.926035    4322 scope.go:122] "RemoveContainer" containerID="01e000d020e94d579be9ba7281fb96cd6a2a1a2263007f62e58b7619166faf42"
Oct 31 07:40:44.933048 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.933028    4322 scope.go:122] "RemoveContainer" containerID="0b855d77eec48c3fc9963475760e651a6bd1ef3f8da5fb0b0992e6459191119a"
Oct 31 07:40:44.940023 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.940007    4322 scope.go:122] "RemoveContainer" containerID="dbb180cb4c5067a74ac32ccb2afed7bcb5d0d4e87d280f10c224723f84489758"
Oct 31 07:40:44.940306 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:40:44.940287    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbb180cb4c5067a74ac32ccb2afed7bcb5d0d4e87d280f10c224723f84489758\": container with ID starting with dbb180cb4c5067a74ac32ccb2afed7bcb5d0d4e87d280f10c224723f84489758 not found: ID does not exist" containerID="dbb180cb4c5067a74ac32ccb2afed7bcb5d0d4e87d280f10c224723f84489758"
Oct 31 07:40:44.940384 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.940318    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbb180cb4c5067a74ac32ccb2afed7bcb5d0d4e87d280f10c224723f84489758"} err="failed to get container status \"dbb180cb4c5067a74ac32ccb2afed7bcb5d0d4e87d280f10c224723f84489758\": rpc error: code = NotFound desc = could not find container \"dbb180cb4c5067a74ac32ccb2afed7bcb5d0d4e87d280f10c224723f84489758\": container with ID starting with dbb180cb4c5067a74ac32ccb2afed7bcb5d0d4e87d280f10c224723f84489758 not found: ID does not exist"
Oct 31 07:40:44.940384 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.940335    4322 scope.go:122] "RemoveContainer" containerID="0dc698f062ea50f2bc3ee3d520017e1ac2129ffcbc7e74cfde43fdc6157c8480"
Oct 31 07:40:44.940628 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:40:44.940610    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dc698f062ea50f2bc3ee3d520017e1ac2129ffcbc7e74cfde43fdc6157c8480\": container with ID starting with 0dc698f062ea50f2bc3ee3d520017e1ac2129ffcbc7e74cfde43fdc6157c8480 not found: ID does not exist" containerID="0dc698f062ea50f2bc3ee3d520017e1ac2129ffcbc7e74cfde43fdc6157c8480"
Oct 31 07:40:44.940695 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.940633    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dc698f062ea50f2bc3ee3d520017e1ac2129ffcbc7e74cfde43fdc6157c8480"} err="failed to get container status \"0dc698f062ea50f2bc3ee3d520017e1ac2129ffcbc7e74cfde43fdc6157c8480\": rpc error: code = NotFound desc = could not find container \"0dc698f062ea50f2bc3ee3d520017e1ac2129ffcbc7e74cfde43fdc6157c8480\": container with ID starting with 0dc698f062ea50f2bc3ee3d520017e1ac2129ffcbc7e74cfde43fdc6157c8480 not found: ID does not exist"
Oct 31 07:40:44.940695 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.940650    4322 scope.go:122] "RemoveContainer" containerID="01e000d020e94d579be9ba7281fb96cd6a2a1a2263007f62e58b7619166faf42"
Oct 31 07:40:44.940904 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:40:44.940887    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01e000d020e94d579be9ba7281fb96cd6a2a1a2263007f62e58b7619166faf42\": container with ID starting with 01e000d020e94d579be9ba7281fb96cd6a2a1a2263007f62e58b7619166faf42 not found: ID does not exist" containerID="01e000d020e94d579be9ba7281fb96cd6a2a1a2263007f62e58b7619166faf42"
Oct 31 07:40:44.940974 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.940905    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01e000d020e94d579be9ba7281fb96cd6a2a1a2263007f62e58b7619166faf42"} err="failed to get container status \"01e000d020e94d579be9ba7281fb96cd6a2a1a2263007f62e58b7619166faf42\": rpc error: code = NotFound desc = could not find container \"01e000d020e94d579be9ba7281fb96cd6a2a1a2263007f62e58b7619166faf42\": container with ID starting with 01e000d020e94d579be9ba7281fb96cd6a2a1a2263007f62e58b7619166faf42 not found: ID does not exist"
Oct 31 07:40:44.940974 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.940919    4322 scope.go:122] "RemoveContainer" containerID="0b855d77eec48c3fc9963475760e651a6bd1ef3f8da5fb0b0992e6459191119a"
Oct 31 07:40:44.941147 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:40:44.941125    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b855d77eec48c3fc9963475760e651a6bd1ef3f8da5fb0b0992e6459191119a\": container with ID starting with 0b855d77eec48c3fc9963475760e651a6bd1ef3f8da5fb0b0992e6459191119a not found: ID does not exist" containerID="0b855d77eec48c3fc9963475760e651a6bd1ef3f8da5fb0b0992e6459191119a"
Oct 31 07:40:44.941200 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.941155    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b855d77eec48c3fc9963475760e651a6bd1ef3f8da5fb0b0992e6459191119a"} err="failed to get container status \"0b855d77eec48c3fc9963475760e651a6bd1ef3f8da5fb0b0992e6459191119a\": rpc error: code = NotFound desc = could not find container \"0b855d77eec48c3fc9963475760e651a6bd1ef3f8da5fb0b0992e6459191119a\": container with ID starting with 0b855d77eec48c3fc9963475760e651a6bd1ef3f8da5fb0b0992e6459191119a not found: ID does not exist"
Oct 31 07:40:44.961329 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.961305    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"]
Oct 31 07:40:44.964903 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:44.964882    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-raw-sklearn-batcher-98f08-predictor-7b949bb469-8jx74"]
Oct 31 07:40:46.547902 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.547866    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"]
Oct 31 07:40:46.548297 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548261    4322 cpu_manager.go:414] "containerMap: removing container" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container"
Oct 31 07:40:46.548297 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548274    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container"
Oct 31 07:40:46.548297 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548284    4322 cpu_manager.go:414] "containerMap: removing container" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kube-rbac-proxy"
Oct 31 07:40:46.548297 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548289    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kube-rbac-proxy"
Oct 31 07:40:46.548297 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548300    4322 cpu_manager.go:414] "containerMap: removing container" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="storage-initializer"
Oct 31 07:40:46.548534 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548306    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="storage-initializer"
Oct 31 07:40:46.548534 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548329    4322 cpu_manager.go:414] "containerMap: removing container" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="storage-initializer"
Oct 31 07:40:46.548534 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548334    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="storage-initializer"
Oct 31 07:40:46.548534 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548340    4322 cpu_manager.go:414] "containerMap: removing container" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kserve-container"
Oct 31 07:40:46.548534 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548345    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kserve-container"
Oct 31 07:40:46.548534 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548358    4322 cpu_manager.go:414] "containerMap: removing container" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent"
Oct 31 07:40:46.548534 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548364    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent"
Oct 31 07:40:46.548534 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548378    4322 cpu_manager.go:414] "containerMap: removing container" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kube-rbac-proxy"
Oct 31 07:40:46.548534 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548384    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kube-rbac-proxy"
Oct 31 07:40:46.548534 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548444    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kserve-container"
Oct 31 07:40:46.548534 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548452    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kube-rbac-proxy"
Oct 31 07:40:46.548534 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548462    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="efa5e541-06f0-4ece-a948-5e3037e225de" containerName="kserve-container"
Oct 31 07:40:46.548534 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548469    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="agent"
Oct 31 07:40:46.548534 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.548491    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" containerName="kube-rbac-proxy"
Oct 31 07:40:46.553489 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.553470    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.567888 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.567866    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"seaweedfs-s3-creds\""
Oct 31 07:40:46.579724 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.579699    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"isvc-sklearn-graph-raw-1deaa-kube-rbac-proxy-sar-config\""
Oct 31 07:40:46.579874 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.579730    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"isvc-sklearn-graph-raw-1deaa-predictor-serving-cert\""
Oct 31 07:40:46.579874 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.579790    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"kube-root-ca.crt\""
Oct 31 07:40:46.580088 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.580066    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"openshift-service-ca.crt\""
Oct 31 07:40:46.580183 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.580106    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"default-dockercfg-d6nzh\""
Oct 31 07:40:46.580183 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.580121    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 07:40:46.581856 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.581834    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"]
Oct 31 07:40:46.638426 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.638391    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/25bad883-7c4e-459f-a0e8-d14ecde66cca-proxy-tls\") pod \"isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.638611 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.638457    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/25bad883-7c4e-459f-a0e8-d14ecde66cca-cabundle-cert\") pod \"isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.638611 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.638545    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqswv\" (UniqueName: \"kubernetes.io/projected/25bad883-7c4e-459f-a0e8-d14ecde66cca-kube-api-access-wqswv\") pod \"isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.638611 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.638581    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/25bad883-7c4e-459f-a0e8-d14ecde66cca-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.638611 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.638607    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/25bad883-7c4e-459f-a0e8-d14ecde66cca-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.739097 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.739066    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqswv\" (UniqueName: \"kubernetes.io/projected/25bad883-7c4e-459f-a0e8-d14ecde66cca-kube-api-access-wqswv\") pod \"isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.739321 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.739107    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/25bad883-7c4e-459f-a0e8-d14ecde66cca-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.739321 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.739256    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/25bad883-7c4e-459f-a0e8-d14ecde66cca-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.739456 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.739364    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/25bad883-7c4e-459f-a0e8-d14ecde66cca-proxy-tls\") pod \"isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.739456 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.739415    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/25bad883-7c4e-459f-a0e8-d14ecde66cca-cabundle-cert\") pod \"isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.739671 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.739647    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/25bad883-7c4e-459f-a0e8-d14ecde66cca-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.739852 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.739830    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/25bad883-7c4e-459f-a0e8-d14ecde66cca-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.739980 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.739962    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/25bad883-7c4e-459f-a0e8-d14ecde66cca-cabundle-cert\") pod \"isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.741689 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.741663    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/25bad883-7c4e-459f-a0e8-d14ecde66cca-proxy-tls\") pod \"isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.762370 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.762340    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqswv\" (UniqueName: \"kubernetes.io/projected/25bad883-7c4e-459f-a0e8-d14ecde66cca-kube-api-access-wqswv\") pod \"isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.870294 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.870194    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0c8980e-1f7e-480b-a633-e7317a35b7e4" path="/var/lib/kubelet/pods/f0c8980e-1f7e-480b-a633-e7317a35b7e4/volumes"
Oct 31 07:40:46.870713 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.870697    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:46.947006 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.946974    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"]
Oct 31 07:40:46.954215 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.954191    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:46.969346 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.969322    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"isvc-xgboost-graph-raw-1deaa-kube-rbac-proxy-sar-config\""
Oct 31 07:40:46.969448 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.969357    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"isvc-xgboost-graph-raw-1deaa-predictor-serving-cert\""
Oct 31 07:40:46.975838 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:46.975810    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"]
Oct 31 07:40:47.042045 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.042014    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hk84\" (UniqueName: \"kubernetes.io/projected/f3874750-754d-40dc-8ef1-a214041623a6-kube-api-access-5hk84\") pod \"isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.042212 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.042068    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f3874750-754d-40dc-8ef1-a214041623a6-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.042212 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.042095    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f3874750-754d-40dc-8ef1-a214041623a6-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.042212 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.042171    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f3874750-754d-40dc-8ef1-a214041623a6-cabundle-cert\") pod \"isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.042370 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.042293    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f3874750-754d-40dc-8ef1-a214041623a6-proxy-tls\") pod \"isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.143244 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.143155    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f3874750-754d-40dc-8ef1-a214041623a6-proxy-tls\") pod \"isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.143396 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.143248    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hk84\" (UniqueName: \"kubernetes.io/projected/f3874750-754d-40dc-8ef1-a214041623a6-kube-api-access-5hk84\") pod \"isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.143396 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.143285    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f3874750-754d-40dc-8ef1-a214041623a6-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.143396 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.143301    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f3874750-754d-40dc-8ef1-a214041623a6-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.143396 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.143315    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f3874750-754d-40dc-8ef1-a214041623a6-cabundle-cert\") pod \"isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.143729 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.143708    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f3874750-754d-40dc-8ef1-a214041623a6-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.143975 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.143957    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f3874750-754d-40dc-8ef1-a214041623a6-cabundle-cert\") pod \"isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.144152 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.144127    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f3874750-754d-40dc-8ef1-a214041623a6-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.145607 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.145590    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f3874750-754d-40dc-8ef1-a214041623a6-proxy-tls\") pod \"isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.166754 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.166732    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hk84\" (UniqueName: \"kubernetes.io/projected/f3874750-754d-40dc-8ef1-a214041623a6-kube-api-access-5hk84\") pod \"isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.176329 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.176301    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"]
Oct 31 07:40:47.179819 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:40:47.179794    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25bad883_7c4e_459f_a0e8_d14ecde66cca.slice/crio-eddf041842ab57667926598e7df399cfd88a26c28c8951fb5567f0ce982294ae WatchSource:0}: Error finding container eddf041842ab57667926598e7df399cfd88a26c28c8951fb5567f0ce982294ae: Status 404 returned error can't find the container with id eddf041842ab57667926598e7df399cfd88a26c28c8951fb5567f0ce982294ae
Oct 31 07:40:47.271824 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.271801    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:40:47.574124 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.574087    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"]
Oct 31 07:40:47.578174 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:40:47.578145    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3874750_754d_40dc_8ef1_a214041623a6.slice/crio-5d6d94e166abce7c4a854378577c44c50dede68c500d9179342f177776446edb WatchSource:0}: Error finding container 5d6d94e166abce7c4a854378577c44c50dede68c500d9179342f177776446edb: Status 404 returned error can't find the container with id 5d6d94e166abce7c4a854378577c44c50dede68c500d9179342f177776446edb
Oct 31 07:40:47.928350 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.928258    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" event={"ID":"25bad883-7c4e-459f-a0e8-d14ecde66cca","Type":"ContainerStarted","Data":"f3e03ec8d9749cb77ce321414366e6fc52244d926c8ebcfff9a2a01cbb0f7bb7"}
Oct 31 07:40:47.928350 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.928325    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" event={"ID":"25bad883-7c4e-459f-a0e8-d14ecde66cca","Type":"ContainerStarted","Data":"eddf041842ab57667926598e7df399cfd88a26c28c8951fb5567f0ce982294ae"}
Oct 31 07:40:47.934897 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.934807    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" event={"ID":"f3874750-754d-40dc-8ef1-a214041623a6","Type":"ContainerStarted","Data":"28f7d52fd73f4b25218aaa9db2d9136bcb9f443b44ed91e94ee3857ff7ae24e4"}
Oct 31 07:40:47.934897 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:47.934873    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" event={"ID":"f3874750-754d-40dc-8ef1-a214041623a6","Type":"ContainerStarted","Data":"5d6d94e166abce7c4a854378577c44c50dede68c500d9179342f177776446edb"}
Oct 31 07:40:51.788519 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:51.788487    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 31 07:40:51.954007 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:51.953976    4322 generic.go:356] "Generic (PLEG): container finished" podID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerID="f3e03ec8d9749cb77ce321414366e6fc52244d926c8ebcfff9a2a01cbb0f7bb7" exitCode=0
Oct 31 07:40:51.954182 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:51.954044    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" event={"ID":"25bad883-7c4e-459f-a0e8-d14ecde66cca","Type":"ContainerDied","Data":"f3e03ec8d9749cb77ce321414366e6fc52244d926c8ebcfff9a2a01cbb0f7bb7"}
Oct 31 07:40:51.955816 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:51.955792    4322 generic.go:356] "Generic (PLEG): container finished" podID="f3874750-754d-40dc-8ef1-a214041623a6" containerID="28f7d52fd73f4b25218aaa9db2d9136bcb9f443b44ed91e94ee3857ff7ae24e4" exitCode=0
Oct 31 07:40:51.955925 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:51.955822    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" event={"ID":"f3874750-754d-40dc-8ef1-a214041623a6","Type":"ContainerDied","Data":"28f7d52fd73f4b25218aaa9db2d9136bcb9f443b44ed91e94ee3857ff7ae24e4"}
Oct 31 07:40:51.957009 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:51.956994    4322 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 07:40:52.964894 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:52.964853    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" event={"ID":"25bad883-7c4e-459f-a0e8-d14ecde66cca","Type":"ContainerStarted","Data":"6bf7d2704b06fb4780d092a0d74274c066e584718186f5527fdc00cd0356bcd2"}
Oct 31 07:40:52.965393 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:52.964915    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" event={"ID":"25bad883-7c4e-459f-a0e8-d14ecde66cca","Type":"ContainerStarted","Data":"1c1d03c6ca96bf38bec7de9fcc5780c4afc16b6dc1a173fe31e45540ba44cac0"}
Oct 31 07:40:52.965393 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:52.965127    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:53.022203 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:53.022139    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" podStartSLOduration=3.196907636 podStartE2EDuration="7.022120128s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.825212492s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:40:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:40:53.019789282 +0000 UTC m=+694.283609892" watchObservedRunningTime="2026-10-31 07:40:53.022120128 +0000 UTC m=+694.285940740"
Oct 31 07:40:53.971978 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:53.971947    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:53.973540 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:53.973512    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 31 07:40:54.976686 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:54.976648    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 31 07:40:59.982142 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:59.982104    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:40:59.982748 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:40:59.982666    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 31 07:41:01.712871 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:41:01.712833    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:41:01.788044 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:01.788006    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 31 07:41:09.983098 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:09.983059    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 31 07:41:11.063117 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:11.063086    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" event={"ID":"f3874750-754d-40dc-8ef1-a214041623a6","Type":"ContainerStarted","Data":"98fc19418178ac0b71bddc918aed4ee471411a6a4d4efdca726bcf10d93584d6"}
Oct 31 07:41:11.787999 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:11.787957    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 31 07:41:12.070555 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:12.070473    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" event={"ID":"f3874750-754d-40dc-8ef1-a214041623a6","Type":"ContainerStarted","Data":"0795cc32a444388ad1e08cfda5f72a8aa52e6094ede897debd2c46a1c347424d"}
Oct 31 07:41:12.070916 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:12.070613    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:41:12.127025 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:12.126966    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" podStartSLOduration=3.018748257 podStartE2EDuration="26.126949371s" totalImagesPullingTime="18.993644459s" totalInitContainerRuntime="4.114556655s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:41:12.125327224 +0000 UTC m=+713.389147834" watchObservedRunningTime="2026-10-31 07:41:12.126949371 +0000 UTC m=+713.390769984"
Oct 31 07:41:13.075099 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:13.075070    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:41:13.076541 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:13.076514    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 31 07:41:14.079569 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:14.079531    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 31 07:41:19.084099 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:19.084070    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:41:19.084583 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:19.084556    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 31 07:41:19.983354 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:19.983314    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 31 07:41:21.788926 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:21.788882    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 31 07:41:29.084825 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:29.084775    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 31 07:41:29.982561 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:29.982522    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 31 07:41:31.710404 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:41:31.710363    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:41:31.788852 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:31.788811    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 31 07:41:39.084622 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:39.084584    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 31 07:41:39.983351 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:39.983314    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 31 07:41:41.788420 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:41.788390    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:41:49.084962 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:49.084920    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 31 07:41:49.359627 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:49.359543    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"]
Oct 31 07:41:49.359911 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:49.359866    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kserve-container" containerID="cri-o://59bdc692835ce52a4b308780b36a2c0964927dac667f353519ecae51587ff24e" gracePeriod=30
Oct 31 07:41:49.359983 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:49.359886    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kube-rbac-proxy" containerID="cri-o://663a00148e8cf1b8b7995e64fd510b3dd88770da872f43796138bfd6eac391c8" gracePeriod=30
Oct 31 07:41:49.982523 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:49.982485    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 31 07:41:50.246266 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:50.246149    4322 generic.go:356] "Generic (PLEG): container finished" podID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerID="663a00148e8cf1b8b7995e64fd510b3dd88770da872f43796138bfd6eac391c8" exitCode=0
Oct 31 07:41:50.246266 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:50.246240    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" event={"ID":"8c19c27a-86bb-448b-bc1b-59d001ca9292","Type":"ContainerDied","Data":"663a00148e8cf1b8b7995e64fd510b3dd88770da872f43796138bfd6eac391c8"}
Oct 31 07:41:51.783012 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:51.782966    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.30:8643/healthz\": dial tcp 10.134.0.30:8643: connect: connection refused"
Oct 31 07:41:51.788335 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:51.788308    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 31 07:41:54.413630 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.413600    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:41:54.547340 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.547305    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c19c27a-86bb-448b-bc1b-59d001ca9292-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c19c27a-86bb-448b-bc1b-59d001ca9292-kube-rbac-proxy-sar-config\") pod \"8c19c27a-86bb-448b-bc1b-59d001ca9292\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") "
Oct 31 07:41:54.547488 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.547376    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c19c27a-86bb-448b-bc1b-59d001ca9292-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8c19c27a-86bb-448b-bc1b-59d001ca9292-cabundle-cert\") pod \"8c19c27a-86bb-448b-bc1b-59d001ca9292\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") "
Oct 31 07:41:54.547488 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.547433    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c19c27a-86bb-448b-bc1b-59d001ca9292-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c19c27a-86bb-448b-bc1b-59d001ca9292-kserve-provision-location\") pod \"8c19c27a-86bb-448b-bc1b-59d001ca9292\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") "
Oct 31 07:41:54.547583 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.547489    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c19c27a-86bb-448b-bc1b-59d001ca9292-kube-api-access-4jbmp\" (UniqueName: \"kubernetes.io/projected/8c19c27a-86bb-448b-bc1b-59d001ca9292-kube-api-access-4jbmp\") pod \"8c19c27a-86bb-448b-bc1b-59d001ca9292\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") "
Oct 31 07:41:54.547583 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.547537    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c19c27a-86bb-448b-bc1b-59d001ca9292-proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c19c27a-86bb-448b-bc1b-59d001ca9292-proxy-tls\") pod \"8c19c27a-86bb-448b-bc1b-59d001ca9292\" (UID: \"8c19c27a-86bb-448b-bc1b-59d001ca9292\") "
Oct 31 07:41:54.547756 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.547730    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c19c27a-86bb-448b-bc1b-59d001ca9292-cabundle-cert" pod "8c19c27a-86bb-448b-bc1b-59d001ca9292" (UID: "8c19c27a-86bb-448b-bc1b-59d001ca9292"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:41:54.547850 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.547717    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c19c27a-86bb-448b-bc1b-59d001ca9292-kube-rbac-proxy-sar-config" pod "8c19c27a-86bb-448b-bc1b-59d001ca9292" (UID: "8c19c27a-86bb-448b-bc1b-59d001ca9292"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:41:54.547850 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.547817    4322 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8c19c27a-86bb-448b-bc1b-59d001ca9292-cabundle-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:41:54.547954 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.547852    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c19c27a-86bb-448b-bc1b-59d001ca9292-kserve-provision-location" pod "8c19c27a-86bb-448b-bc1b-59d001ca9292" (UID: "8c19c27a-86bb-448b-bc1b-59d001ca9292"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:41:54.549651 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.549620    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c19c27a-86bb-448b-bc1b-59d001ca9292-proxy-tls" pod "8c19c27a-86bb-448b-bc1b-59d001ca9292" (UID: "8c19c27a-86bb-448b-bc1b-59d001ca9292"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:41:54.549757 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.549676    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c19c27a-86bb-448b-bc1b-59d001ca9292-kube-api-access-4jbmp" pod "8c19c27a-86bb-448b-bc1b-59d001ca9292" (UID: "8c19c27a-86bb-448b-bc1b-59d001ca9292"). InnerVolumeSpecName "kube-api-access-4jbmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:41:54.648426 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.648387    4322 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c19c27a-86bb-448b-bc1b-59d001ca9292-kserve-provision-location\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:41:54.648426 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.648429    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jbmp\" (UniqueName: \"kubernetes.io/projected/8c19c27a-86bb-448b-bc1b-59d001ca9292-kube-api-access-4jbmp\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:41:54.648614 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.648445    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c19c27a-86bb-448b-bc1b-59d001ca9292-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:41:54.648614 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:54.648460    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c19c27a-86bb-448b-bc1b-59d001ca9292-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:41:55.270365 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:55.270331    4322 generic.go:356] "Generic (PLEG): container finished" podID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerID="59bdc692835ce52a4b308780b36a2c0964927dac667f353519ecae51587ff24e" exitCode=0
Oct 31 07:41:55.270579 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:55.270396    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"
Oct 31 07:41:55.270579 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:55.270422    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" event={"ID":"8c19c27a-86bb-448b-bc1b-59d001ca9292","Type":"ContainerDied","Data":"59bdc692835ce52a4b308780b36a2c0964927dac667f353519ecae51587ff24e"}
Oct 31 07:41:55.270579 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:55.270503    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw" event={"ID":"8c19c27a-86bb-448b-bc1b-59d001ca9292","Type":"ContainerDied","Data":"806efa4aff6573e147d7d4dd0dfd0793c6269c4e4883dd941b4141bfab9a0191"}
Oct 31 07:41:55.270579 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:55.270562    4322 scope.go:122] "RemoveContainer" containerID="663a00148e8cf1b8b7995e64fd510b3dd88770da872f43796138bfd6eac391c8"
Oct 31 07:41:55.278946 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:55.278777    4322 scope.go:122] "RemoveContainer" containerID="59bdc692835ce52a4b308780b36a2c0964927dac667f353519ecae51587ff24e"
Oct 31 07:41:55.286135 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:55.286107    4322 scope.go:122] "RemoveContainer" containerID="cc082cead0978ce4f63de13c4648df99f5803b82352ede85844febe49988a6a7"
Oct 31 07:41:55.293257 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:55.293241    4322 scope.go:122] "RemoveContainer" containerID="663a00148e8cf1b8b7995e64fd510b3dd88770da872f43796138bfd6eac391c8"
Oct 31 07:41:55.293552 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:41:55.293530    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"663a00148e8cf1b8b7995e64fd510b3dd88770da872f43796138bfd6eac391c8\": container with ID starting with 663a00148e8cf1b8b7995e64fd510b3dd88770da872f43796138bfd6eac391c8 not found: ID does not exist" containerID="663a00148e8cf1b8b7995e64fd510b3dd88770da872f43796138bfd6eac391c8"
Oct 31 07:41:55.293606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:55.293563    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"663a00148e8cf1b8b7995e64fd510b3dd88770da872f43796138bfd6eac391c8"} err="failed to get container status \"663a00148e8cf1b8b7995e64fd510b3dd88770da872f43796138bfd6eac391c8\": rpc error: code = NotFound desc = could not find container \"663a00148e8cf1b8b7995e64fd510b3dd88770da872f43796138bfd6eac391c8\": container with ID starting with 663a00148e8cf1b8b7995e64fd510b3dd88770da872f43796138bfd6eac391c8 not found: ID does not exist"
Oct 31 07:41:55.293606 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:55.293580    4322 scope.go:122] "RemoveContainer" containerID="59bdc692835ce52a4b308780b36a2c0964927dac667f353519ecae51587ff24e"
Oct 31 07:41:55.293831 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:41:55.293794    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59bdc692835ce52a4b308780b36a2c0964927dac667f353519ecae51587ff24e\": container with ID starting with 59bdc692835ce52a4b308780b36a2c0964927dac667f353519ecae51587ff24e not found: ID does not exist" containerID="59bdc692835ce52a4b308780b36a2c0964927dac667f353519ecae51587ff24e"
Oct 31 07:41:55.293897 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:55.293840    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59bdc692835ce52a4b308780b36a2c0964927dac667f353519ecae51587ff24e"} err="failed to get container status \"59bdc692835ce52a4b308780b36a2c0964927dac667f353519ecae51587ff24e\": rpc error: code = NotFound desc = could not find container \"59bdc692835ce52a4b308780b36a2c0964927dac667f353519ecae51587ff24e\": container with ID starting with 59bdc692835ce52a4b308780b36a2c0964927dac667f353519ecae51587ff24e not found: ID does not exist"
Oct 31 07:41:55.293897 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:55.293865    4322 scope.go:122] "RemoveContainer" containerID="cc082cead0978ce4f63de13c4648df99f5803b82352ede85844febe49988a6a7"
Oct 31 07:41:55.294092 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:41:55.294072    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc082cead0978ce4f63de13c4648df99f5803b82352ede85844febe49988a6a7\": container with ID starting with cc082cead0978ce4f63de13c4648df99f5803b82352ede85844febe49988a6a7 not found: ID does not exist" containerID="cc082cead0978ce4f63de13c4648df99f5803b82352ede85844febe49988a6a7"
Oct 31 07:41:55.294136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:55.294098    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc082cead0978ce4f63de13c4648df99f5803b82352ede85844febe49988a6a7"} err="failed to get container status \"cc082cead0978ce4f63de13c4648df99f5803b82352ede85844febe49988a6a7\": rpc error: code = NotFound desc = could not find container \"cc082cead0978ce4f63de13c4648df99f5803b82352ede85844febe49988a6a7\": container with ID starting with cc082cead0978ce4f63de13c4648df99f5803b82352ede85844febe49988a6a7 not found: ID does not exist"
Oct 31 07:41:55.333354 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:55.333325    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"]
Oct 31 07:41:55.344954 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:55.344922    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-rollback-predictor-5b7b487756-6tdtw"]
Oct 31 07:41:56.870941 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:56.870897    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" path="/var/lib/kubelet/pods/8c19c27a-86bb-448b-bc1b-59d001ca9292/volumes"
Oct 31 07:41:57.269240 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.269203    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"]
Oct 31 07:41:57.269689 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.269665    4322 cpu_manager.go:414] "containerMap: removing container" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="storage-initializer"
Oct 31 07:41:57.269689 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.269688    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="storage-initializer"
Oct 31 07:41:57.269885 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.269719    4322 cpu_manager.go:414] "containerMap: removing container" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kube-rbac-proxy"
Oct 31 07:41:57.269885 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.269728    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kube-rbac-proxy"
Oct 31 07:41:57.269885 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.269746    4322 cpu_manager.go:414] "containerMap: removing container" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kserve-container"
Oct 31 07:41:57.269885 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.269753    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kserve-container"
Oct 31 07:41:57.269885 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.269852    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kube-rbac-proxy"
Oct 31 07:41:57.269885 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.269874    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c19c27a-86bb-448b-bc1b-59d001ca9292" containerName="kserve-container"
Oct 31 07:41:57.274905 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.274886    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.289448 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.289419    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7281997251\"/\"isvc-canary-stop-predictor-serving-cert\""
Oct 31 07:41:57.289448 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.289438    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"isvc-canary-stop-kube-rbac-proxy-sar-config\""
Oct 31 07:41:57.289615 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.289438    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"openshift-service-ca.crt\""
Oct 31 07:41:57.289615 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.289442    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 07:41:57.289615 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.289452    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7281997251\"/\"seaweedfs-s3-creds\""
Oct 31 07:41:57.289615 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.289460    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"kube-root-ca.crt\""
Oct 31 07:41:57.304100 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.301462    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7281997251\"/\"default-dockercfg-t4zp9\""
Oct 31 07:41:57.304100 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.303000    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"]
Oct 31 07:41:57.370643 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.370609    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kserve-provision-location\") pod \"isvc-canary-stop-predictor-589d977cbd-fq75c\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") " pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.370818 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.370683    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-cabundle-cert\") pod \"isvc-canary-stop-predictor-589d977cbd-fq75c\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") " pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.370818 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.370788    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-proxy-tls\") pod \"isvc-canary-stop-predictor-589d977cbd-fq75c\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") " pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.370928 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.370827    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vq9j\" (UniqueName: \"kubernetes.io/projected/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kube-api-access-7vq9j\") pod \"isvc-canary-stop-predictor-589d977cbd-fq75c\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") " pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.370928 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.370858    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-589d977cbd-fq75c\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") " pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.471581 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.471541    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kserve-provision-location\") pod \"isvc-canary-stop-predictor-589d977cbd-fq75c\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") " pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.471730 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.471711    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-cabundle-cert\") pod \"isvc-canary-stop-predictor-589d977cbd-fq75c\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") " pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.471785 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.471763    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-proxy-tls\") pod \"isvc-canary-stop-predictor-589d977cbd-fq75c\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") " pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.471785 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.471782    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vq9j\" (UniqueName: \"kubernetes.io/projected/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kube-api-access-7vq9j\") pod \"isvc-canary-stop-predictor-589d977cbd-fq75c\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") " pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.471853 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.471802    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-589d977cbd-fq75c\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") " pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.471996 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.471970    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kserve-provision-location\") pod \"isvc-canary-stop-predictor-589d977cbd-fq75c\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") " pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.472504 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.472479    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-589d977cbd-fq75c\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") " pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.472572 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.472476    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-cabundle-cert\") pod \"isvc-canary-stop-predictor-589d977cbd-fq75c\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") " pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.474309 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.474290    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-proxy-tls\") pod \"isvc-canary-stop-predictor-589d977cbd-fq75c\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") " pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.494853 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.494830    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vq9j\" (UniqueName: \"kubernetes.io/projected/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kube-api-access-7vq9j\") pod \"isvc-canary-stop-predictor-589d977cbd-fq75c\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") " pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.593089 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.593006    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:41:57.900269 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:57.900053    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"]
Oct 31 07:41:57.903198 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:41:57.903173    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6ea81a4_8199_49cc_bca8_8dfa6b2dbccb.slice/crio-f99a5a3a68d259a27eea4170f1cf618e558594807bb0224283f5c8efe59ebfd7 WatchSource:0}: Error finding container f99a5a3a68d259a27eea4170f1cf618e558594807bb0224283f5c8efe59ebfd7: Status 404 returned error can't find the container with id f99a5a3a68d259a27eea4170f1cf618e558594807bb0224283f5c8efe59ebfd7
Oct 31 07:41:58.287384 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:58.287347    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" event={"ID":"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb","Type":"ContainerStarted","Data":"7c7bab6aac0b1a266c6ebc5b0d649ca4a88f04cfd1ba7d547eb3f1ab3eb03a9f"}
Oct 31 07:41:58.287567 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:58.287410    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" event={"ID":"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb","Type":"ContainerStarted","Data":"f99a5a3a68d259a27eea4170f1cf618e558594807bb0224283f5c8efe59ebfd7"}
Oct 31 07:41:59.085454 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:59.085414    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 31 07:41:59.983339 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:41:59.983300    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 31 07:42:01.710999 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:42:01.710919    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:42:02.306018 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:02.305984    4322 generic.go:356] "Generic (PLEG): container finished" podID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerID="7c7bab6aac0b1a266c6ebc5b0d649ca4a88f04cfd1ba7d547eb3f1ab3eb03a9f" exitCode=0
Oct 31 07:42:02.306175 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:02.306059    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" event={"ID":"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb","Type":"ContainerDied","Data":"7c7bab6aac0b1a266c6ebc5b0d649ca4a88f04cfd1ba7d547eb3f1ab3eb03a9f"}
Oct 31 07:42:03.312372 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:03.312338    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" event={"ID":"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb","Type":"ContainerStarted","Data":"b37a9c8fb3374eed9a9262e923900642cc64cfdbfc406374e53c33a3ba622b37"}
Oct 31 07:42:03.312768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:03.312398    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" event={"ID":"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb","Type":"ContainerStarted","Data":"cff95a48dd124eee6383c46a19c9392c5879c1b0c34a75a99b157cc058bb6138"}
Oct 31 07:42:03.312768 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:03.312584    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:42:03.370659 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:03.370613    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" podStartSLOduration=2.349058785 podStartE2EDuration="6.37060043s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.021541645s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:41:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:42:03.36921387 +0000 UTC m=+764.633034481" watchObservedRunningTime="2026-10-31 07:42:03.37060043 +0000 UTC m=+764.634421040"
Oct 31 07:42:04.316836 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:04.316805    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:42:04.318143 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:04.318115    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 31 07:42:05.321180 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:05.321139    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 31 07:42:09.084671 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:09.084628    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 31 07:42:09.982548 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:09.982507    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 31 07:42:10.325700 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:10.325669    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:42:10.326135 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:10.326110    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 31 07:42:19.085010 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:19.084966    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 31 07:42:19.983382 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:19.983344    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:42:20.326207 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:20.326164    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 31 07:42:29.085177 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:29.085145    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:42:30.326392 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:30.326352    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 31 07:42:31.710188 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:42:31.710151    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:42:40.326353 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:40.326314    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 31 07:42:47.228110 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:47.228069    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"]
Oct 31 07:42:47.228572 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:47.228376    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kserve-container" containerID="cri-o://1c1d03c6ca96bf38bec7de9fcc5780c4afc16b6dc1a173fe31e45540ba44cac0" gracePeriod=30
Oct 31 07:42:47.228572 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:47.228386    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kube-rbac-proxy" containerID="cri-o://6bf7d2704b06fb4780d092a0d74274c066e584718186f5527fdc00cd0356bcd2" gracePeriod=30
Oct 31 07:42:47.327742 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:47.327702    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"]
Oct 31 07:42:47.328063 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:47.328003    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kserve-container" containerID="cri-o://98fc19418178ac0b71bddc918aed4ee471411a6a4d4efdca726bcf10d93584d6" gracePeriod=30
Oct 31 07:42:47.328214 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:47.328180    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kube-rbac-proxy" containerID="cri-o://0795cc32a444388ad1e08cfda5f72a8aa52e6094ede897debd2c46a1c347424d" gracePeriod=30
Oct 31 07:42:47.507142 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:47.507068    4322 generic.go:356] "Generic (PLEG): container finished" podID="f3874750-754d-40dc-8ef1-a214041623a6" containerID="0795cc32a444388ad1e08cfda5f72a8aa52e6094ede897debd2c46a1c347424d" exitCode=0
Oct 31 07:42:47.507306 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:47.507141    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" event={"ID":"f3874750-754d-40dc-8ef1-a214041623a6","Type":"ContainerDied","Data":"0795cc32a444388ad1e08cfda5f72a8aa52e6094ede897debd2c46a1c347424d"}
Oct 31 07:42:47.510417 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:47.510394    4322 generic.go:356] "Generic (PLEG): container finished" podID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerID="6bf7d2704b06fb4780d092a0d74274c066e584718186f5527fdc00cd0356bcd2" exitCode=0
Oct 31 07:42:47.510417 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:47.510421    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" event={"ID":"25bad883-7c4e-459f-a0e8-d14ecde66cca","Type":"ContainerDied","Data":"6bf7d2704b06fb4780d092a0d74274c066e584718186f5527fdc00cd0356bcd2"}
Oct 31 07:42:49.079781 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:49.079731    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.32:8643/healthz\": dial tcp 10.134.0.32:8643: connect: connection refused"
Oct 31 07:42:49.085068 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:49.085044    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 31 07:42:49.977080 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:49.977035    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.31:8643/healthz\": dial tcp 10.134.0.31:8643: connect: connection refused"
Oct 31 07:42:49.983455 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:49.983425    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 31 07:42:50.326994 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:50.326959    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 31 07:42:51.372005 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.371978    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:42:51.530965 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.530931    4322 generic.go:356] "Generic (PLEG): container finished" podID="f3874750-754d-40dc-8ef1-a214041623a6" containerID="98fc19418178ac0b71bddc918aed4ee471411a6a4d4efdca726bcf10d93584d6" exitCode=0
Oct 31 07:42:51.531127 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.530975    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" event={"ID":"f3874750-754d-40dc-8ef1-a214041623a6","Type":"ContainerDied","Data":"98fc19418178ac0b71bddc918aed4ee471411a6a4d4efdca726bcf10d93584d6"}
Oct 31 07:42:51.531127 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.530990    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"
Oct 31 07:42:51.531127 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.531038    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg" event={"ID":"f3874750-754d-40dc-8ef1-a214041623a6","Type":"ContainerDied","Data":"5d6d94e166abce7c4a854378577c44c50dede68c500d9179342f177776446edb"}
Oct 31 07:42:51.531127 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.531068    4322 scope.go:122] "RemoveContainer" containerID="0795cc32a444388ad1e08cfda5f72a8aa52e6094ede897debd2c46a1c347424d"
Oct 31 07:42:51.539330 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.539306    4322 scope.go:122] "RemoveContainer" containerID="98fc19418178ac0b71bddc918aed4ee471411a6a4d4efdca726bcf10d93584d6"
Oct 31 07:42:51.545504 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.545480    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f3874750-754d-40dc-8ef1-a214041623a6-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f3874750-754d-40dc-8ef1-a214041623a6-kube-rbac-proxy-sar-config\") pod \"f3874750-754d-40dc-8ef1-a214041623a6\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") "
Oct 31 07:42:51.545753 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.545733    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f3874750-754d-40dc-8ef1-a214041623a6-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f3874750-754d-40dc-8ef1-a214041623a6-cabundle-cert\") pod \"f3874750-754d-40dc-8ef1-a214041623a6\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") "
Oct 31 07:42:51.545834 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.545784    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3874750-754d-40dc-8ef1-a214041623a6-kube-api-access-5hk84\" (UniqueName: \"kubernetes.io/projected/f3874750-754d-40dc-8ef1-a214041623a6-kube-api-access-5hk84\") pod \"f3874750-754d-40dc-8ef1-a214041623a6\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") "
Oct 31 07:42:51.545834 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.545825    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3874750-754d-40dc-8ef1-a214041623a6-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f3874750-754d-40dc-8ef1-a214041623a6-kserve-provision-location\") pod \"f3874750-754d-40dc-8ef1-a214041623a6\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") "
Oct 31 07:42:51.545941 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.545850    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f3874750-754d-40dc-8ef1-a214041623a6-proxy-tls\" (UniqueName: \"kubernetes.io/secret/f3874750-754d-40dc-8ef1-a214041623a6-proxy-tls\") pod \"f3874750-754d-40dc-8ef1-a214041623a6\" (UID: \"f3874750-754d-40dc-8ef1-a214041623a6\") "
Oct 31 07:42:51.545941 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.545902    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3874750-754d-40dc-8ef1-a214041623a6-kube-rbac-proxy-sar-config" pod "f3874750-754d-40dc-8ef1-a214041623a6" (UID: "f3874750-754d-40dc-8ef1-a214041623a6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:42:51.546129 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.546109    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f3874750-754d-40dc-8ef1-a214041623a6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:42:51.546459 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.546439    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3874750-754d-40dc-8ef1-a214041623a6-kserve-provision-location" pod "f3874750-754d-40dc-8ef1-a214041623a6" (UID: "f3874750-754d-40dc-8ef1-a214041623a6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:42:51.546540 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.546482    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3874750-754d-40dc-8ef1-a214041623a6-cabundle-cert" pod "f3874750-754d-40dc-8ef1-a214041623a6" (UID: "f3874750-754d-40dc-8ef1-a214041623a6"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:42:51.547441 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.547421    4322 scope.go:122] "RemoveContainer" containerID="28f7d52fd73f4b25218aaa9db2d9136bcb9f443b44ed91e94ee3857ff7ae24e4"
Oct 31 07:42:51.548713 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.548445    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3874750-754d-40dc-8ef1-a214041623a6-proxy-tls" pod "f3874750-754d-40dc-8ef1-a214041623a6" (UID: "f3874750-754d-40dc-8ef1-a214041623a6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:42:51.548713 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.548588    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3874750-754d-40dc-8ef1-a214041623a6-kube-api-access-5hk84" pod "f3874750-754d-40dc-8ef1-a214041623a6" (UID: "f3874750-754d-40dc-8ef1-a214041623a6"). InnerVolumeSpecName "kube-api-access-5hk84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:42:51.563717 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.563700    4322 scope.go:122] "RemoveContainer" containerID="0795cc32a444388ad1e08cfda5f72a8aa52e6094ede897debd2c46a1c347424d"
Oct 31 07:42:51.563975 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:42:51.563956    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0795cc32a444388ad1e08cfda5f72a8aa52e6094ede897debd2c46a1c347424d\": container with ID starting with 0795cc32a444388ad1e08cfda5f72a8aa52e6094ede897debd2c46a1c347424d not found: ID does not exist" containerID="0795cc32a444388ad1e08cfda5f72a8aa52e6094ede897debd2c46a1c347424d"
Oct 31 07:42:51.564020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.563986    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0795cc32a444388ad1e08cfda5f72a8aa52e6094ede897debd2c46a1c347424d"} err="failed to get container status \"0795cc32a444388ad1e08cfda5f72a8aa52e6094ede897debd2c46a1c347424d\": rpc error: code = NotFound desc = could not find container \"0795cc32a444388ad1e08cfda5f72a8aa52e6094ede897debd2c46a1c347424d\": container with ID starting with 0795cc32a444388ad1e08cfda5f72a8aa52e6094ede897debd2c46a1c347424d not found: ID does not exist"
Oct 31 07:42:51.564020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.564003    4322 scope.go:122] "RemoveContainer" containerID="98fc19418178ac0b71bddc918aed4ee471411a6a4d4efdca726bcf10d93584d6"
Oct 31 07:42:51.564211 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:42:51.564192    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98fc19418178ac0b71bddc918aed4ee471411a6a4d4efdca726bcf10d93584d6\": container with ID starting with 98fc19418178ac0b71bddc918aed4ee471411a6a4d4efdca726bcf10d93584d6 not found: ID does not exist" containerID="98fc19418178ac0b71bddc918aed4ee471411a6a4d4efdca726bcf10d93584d6"
Oct 31 07:42:51.564275 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.564218    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98fc19418178ac0b71bddc918aed4ee471411a6a4d4efdca726bcf10d93584d6"} err="failed to get container status \"98fc19418178ac0b71bddc918aed4ee471411a6a4d4efdca726bcf10d93584d6\": rpc error: code = NotFound desc = could not find container \"98fc19418178ac0b71bddc918aed4ee471411a6a4d4efdca726bcf10d93584d6\": container with ID starting with 98fc19418178ac0b71bddc918aed4ee471411a6a4d4efdca726bcf10d93584d6 not found: ID does not exist"
Oct 31 07:42:51.564275 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.564249    4322 scope.go:122] "RemoveContainer" containerID="28f7d52fd73f4b25218aaa9db2d9136bcb9f443b44ed91e94ee3857ff7ae24e4"
Oct 31 07:42:51.564469 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:42:51.564452    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28f7d52fd73f4b25218aaa9db2d9136bcb9f443b44ed91e94ee3857ff7ae24e4\": container with ID starting with 28f7d52fd73f4b25218aaa9db2d9136bcb9f443b44ed91e94ee3857ff7ae24e4 not found: ID does not exist" containerID="28f7d52fd73f4b25218aaa9db2d9136bcb9f443b44ed91e94ee3857ff7ae24e4"
Oct 31 07:42:51.564516 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.564476    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28f7d52fd73f4b25218aaa9db2d9136bcb9f443b44ed91e94ee3857ff7ae24e4"} err="failed to get container status \"28f7d52fd73f4b25218aaa9db2d9136bcb9f443b44ed91e94ee3857ff7ae24e4\": rpc error: code = NotFound desc = could not find container \"28f7d52fd73f4b25218aaa9db2d9136bcb9f443b44ed91e94ee3857ff7ae24e4\": container with ID starting with 28f7d52fd73f4b25218aaa9db2d9136bcb9f443b44ed91e94ee3857ff7ae24e4 not found: ID does not exist"
Oct 31 07:42:51.647519 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.647491    4322 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f3874750-754d-40dc-8ef1-a214041623a6-cabundle-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:42:51.647519 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.647516    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hk84\" (UniqueName: \"kubernetes.io/projected/f3874750-754d-40dc-8ef1-a214041623a6-kube-api-access-5hk84\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:42:51.647696 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.647527    4322 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f3874750-754d-40dc-8ef1-a214041623a6-kserve-provision-location\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:42:51.647696 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.647537    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f3874750-754d-40dc-8ef1-a214041623a6-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:42:51.875862 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.875804    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"]
Oct 31 07:42:51.887489 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:51.887461    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-1deaa-predictor-5b9c46688f-8nmkg"]
Oct 31 07:42:52.172568 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.172548    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:42:52.354034 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.353953    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25bad883-7c4e-459f-a0e8-d14ecde66cca-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/25bad883-7c4e-459f-a0e8-d14ecde66cca-kserve-provision-location\") pod \"25bad883-7c4e-459f-a0e8-d14ecde66cca\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") "
Oct 31 07:42:52.354034 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.354003    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25bad883-7c4e-459f-a0e8-d14ecde66cca-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/25bad883-7c4e-459f-a0e8-d14ecde66cca-cabundle-cert\") pod \"25bad883-7c4e-459f-a0e8-d14ecde66cca\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") "
Oct 31 07:42:52.354034 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.354038    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25bad883-7c4e-459f-a0e8-d14ecde66cca-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/25bad883-7c4e-459f-a0e8-d14ecde66cca-kube-rbac-proxy-sar-config\") pod \"25bad883-7c4e-459f-a0e8-d14ecde66cca\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") "
Oct 31 07:42:52.354331 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.354063    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25bad883-7c4e-459f-a0e8-d14ecde66cca-kube-api-access-wqswv\" (UniqueName: \"kubernetes.io/projected/25bad883-7c4e-459f-a0e8-d14ecde66cca-kube-api-access-wqswv\") pod \"25bad883-7c4e-459f-a0e8-d14ecde66cca\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") "
Oct 31 07:42:52.354331 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.354095    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25bad883-7c4e-459f-a0e8-d14ecde66cca-proxy-tls\" (UniqueName: \"kubernetes.io/secret/25bad883-7c4e-459f-a0e8-d14ecde66cca-proxy-tls\") pod \"25bad883-7c4e-459f-a0e8-d14ecde66cca\" (UID: \"25bad883-7c4e-459f-a0e8-d14ecde66cca\") "
Oct 31 07:42:52.354450 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.354330    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25bad883-7c4e-459f-a0e8-d14ecde66cca-kserve-provision-location" pod "25bad883-7c4e-459f-a0e8-d14ecde66cca" (UID: "25bad883-7c4e-459f-a0e8-d14ecde66cca"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:42:52.354450 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.354424    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25bad883-7c4e-459f-a0e8-d14ecde66cca-cabundle-cert" pod "25bad883-7c4e-459f-a0e8-d14ecde66cca" (UID: "25bad883-7c4e-459f-a0e8-d14ecde66cca"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:42:52.354556 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.354481    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25bad883-7c4e-459f-a0e8-d14ecde66cca-kube-rbac-proxy-sar-config" pod "25bad883-7c4e-459f-a0e8-d14ecde66cca" (UID: "25bad883-7c4e-459f-a0e8-d14ecde66cca"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:42:52.356217 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.356189    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25bad883-7c4e-459f-a0e8-d14ecde66cca-proxy-tls" pod "25bad883-7c4e-459f-a0e8-d14ecde66cca" (UID: "25bad883-7c4e-459f-a0e8-d14ecde66cca"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:42:52.356217 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.356217    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25bad883-7c4e-459f-a0e8-d14ecde66cca-kube-api-access-wqswv" pod "25bad883-7c4e-459f-a0e8-d14ecde66cca" (UID: "25bad883-7c4e-459f-a0e8-d14ecde66cca"). InnerVolumeSpecName "kube-api-access-wqswv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:42:52.455668 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.455631    4322 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/25bad883-7c4e-459f-a0e8-d14ecde66cca-kserve-provision-location\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:42:52.455668 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.455660    4322 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/25bad883-7c4e-459f-a0e8-d14ecde66cca-cabundle-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:42:52.455668 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.455671    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/25bad883-7c4e-459f-a0e8-d14ecde66cca-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:42:52.455668 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.455680    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqswv\" (UniqueName: \"kubernetes.io/projected/25bad883-7c4e-459f-a0e8-d14ecde66cca-kube-api-access-wqswv\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:42:52.456130 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.455691    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/25bad883-7c4e-459f-a0e8-d14ecde66cca-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:42:52.537289 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.537258    4322 generic.go:356] "Generic (PLEG): container finished" podID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerID="1c1d03c6ca96bf38bec7de9fcc5780c4afc16b6dc1a173fe31e45540ba44cac0" exitCode=0
Oct 31 07:42:52.537470 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.537317    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"
Oct 31 07:42:52.537470 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.537348    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" event={"ID":"25bad883-7c4e-459f-a0e8-d14ecde66cca","Type":"ContainerDied","Data":"1c1d03c6ca96bf38bec7de9fcc5780c4afc16b6dc1a173fe31e45540ba44cac0"}
Oct 31 07:42:52.537470 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.537458    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928" event={"ID":"25bad883-7c4e-459f-a0e8-d14ecde66cca","Type":"ContainerDied","Data":"eddf041842ab57667926598e7df399cfd88a26c28c8951fb5567f0ce982294ae"}
Oct 31 07:42:52.537604 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.537496    4322 scope.go:122] "RemoveContainer" containerID="6bf7d2704b06fb4780d092a0d74274c066e584718186f5527fdc00cd0356bcd2"
Oct 31 07:42:52.546321 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.546303    4322 scope.go:122] "RemoveContainer" containerID="1c1d03c6ca96bf38bec7de9fcc5780c4afc16b6dc1a173fe31e45540ba44cac0"
Oct 31 07:42:52.553685 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.553659    4322 scope.go:122] "RemoveContainer" containerID="f3e03ec8d9749cb77ce321414366e6fc52244d926c8ebcfff9a2a01cbb0f7bb7"
Oct 31 07:42:52.561090 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.561072    4322 scope.go:122] "RemoveContainer" containerID="6bf7d2704b06fb4780d092a0d74274c066e584718186f5527fdc00cd0356bcd2"
Oct 31 07:42:52.561376 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:42:52.561355    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bf7d2704b06fb4780d092a0d74274c066e584718186f5527fdc00cd0356bcd2\": container with ID starting with 6bf7d2704b06fb4780d092a0d74274c066e584718186f5527fdc00cd0356bcd2 not found: ID does not exist" containerID="6bf7d2704b06fb4780d092a0d74274c066e584718186f5527fdc00cd0356bcd2"
Oct 31 07:42:52.561461 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.561383    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bf7d2704b06fb4780d092a0d74274c066e584718186f5527fdc00cd0356bcd2"} err="failed to get container status \"6bf7d2704b06fb4780d092a0d74274c066e584718186f5527fdc00cd0356bcd2\": rpc error: code = NotFound desc = could not find container \"6bf7d2704b06fb4780d092a0d74274c066e584718186f5527fdc00cd0356bcd2\": container with ID starting with 6bf7d2704b06fb4780d092a0d74274c066e584718186f5527fdc00cd0356bcd2 not found: ID does not exist"
Oct 31 07:42:52.561461 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.561408    4322 scope.go:122] "RemoveContainer" containerID="1c1d03c6ca96bf38bec7de9fcc5780c4afc16b6dc1a173fe31e45540ba44cac0"
Oct 31 07:42:52.561665 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:42:52.561651    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c1d03c6ca96bf38bec7de9fcc5780c4afc16b6dc1a173fe31e45540ba44cac0\": container with ID starting with 1c1d03c6ca96bf38bec7de9fcc5780c4afc16b6dc1a173fe31e45540ba44cac0 not found: ID does not exist" containerID="1c1d03c6ca96bf38bec7de9fcc5780c4afc16b6dc1a173fe31e45540ba44cac0"
Oct 31 07:42:52.561701 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.561672    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c1d03c6ca96bf38bec7de9fcc5780c4afc16b6dc1a173fe31e45540ba44cac0"} err="failed to get container status \"1c1d03c6ca96bf38bec7de9fcc5780c4afc16b6dc1a173fe31e45540ba44cac0\": rpc error: code = NotFound desc = could not find container \"1c1d03c6ca96bf38bec7de9fcc5780c4afc16b6dc1a173fe31e45540ba44cac0\": container with ID starting with 1c1d03c6ca96bf38bec7de9fcc5780c4afc16b6dc1a173fe31e45540ba44cac0 not found: ID does not exist"
Oct 31 07:42:52.561701 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.561688    4322 scope.go:122] "RemoveContainer" containerID="f3e03ec8d9749cb77ce321414366e6fc52244d926c8ebcfff9a2a01cbb0f7bb7"
Oct 31 07:42:52.561921 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:42:52.561906    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3e03ec8d9749cb77ce321414366e6fc52244d926c8ebcfff9a2a01cbb0f7bb7\": container with ID starting with f3e03ec8d9749cb77ce321414366e6fc52244d926c8ebcfff9a2a01cbb0f7bb7 not found: ID does not exist" containerID="f3e03ec8d9749cb77ce321414366e6fc52244d926c8ebcfff9a2a01cbb0f7bb7"
Oct 31 07:42:52.561955 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.561926    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3e03ec8d9749cb77ce321414366e6fc52244d926c8ebcfff9a2a01cbb0f7bb7"} err="failed to get container status \"f3e03ec8d9749cb77ce321414366e6fc52244d926c8ebcfff9a2a01cbb0f7bb7\": rpc error: code = NotFound desc = could not find container \"f3e03ec8d9749cb77ce321414366e6fc52244d926c8ebcfff9a2a01cbb0f7bb7\": container with ID starting with f3e03ec8d9749cb77ce321414366e6fc52244d926c8ebcfff9a2a01cbb0f7bb7 not found: ID does not exist"
Oct 31 07:42:52.586965 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.586940    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"]
Oct 31 07:42:52.598893 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.598871    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-1deaa-predictor-55df4864d9-zt928"]
Oct 31 07:42:52.870698 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.870668    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" path="/var/lib/kubelet/pods/25bad883-7c4e-459f-a0e8-d14ecde66cca/volumes"
Oct 31 07:42:52.871165 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:52.871152    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f3874750-754d-40dc-8ef1-a214041623a6" path="/var/lib/kubelet/pods/f3874750-754d-40dc-8ef1-a214041623a6/volumes"
Oct 31 07:42:54.880341 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880312    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"]
Oct 31 07:42:54.880705 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880686    4322 cpu_manager.go:414] "containerMap: removing container" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kserve-container"
Oct 31 07:42:54.880705 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880699    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kserve-container"
Oct 31 07:42:54.880771 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880715    4322 cpu_manager.go:414] "containerMap: removing container" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kube-rbac-proxy"
Oct 31 07:42:54.880771 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880720    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kube-rbac-proxy"
Oct 31 07:42:54.880771 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880729    4322 cpu_manager.go:414] "containerMap: removing container" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="storage-initializer"
Oct 31 07:42:54.880771 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880733    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="storage-initializer"
Oct 31 07:42:54.880771 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880744    4322 cpu_manager.go:414] "containerMap: removing container" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kserve-container"
Oct 31 07:42:54.880771 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880750    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kserve-container"
Oct 31 07:42:54.880771 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880762    4322 cpu_manager.go:414] "containerMap: removing container" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="storage-initializer"
Oct 31 07:42:54.880771 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880768    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="storage-initializer"
Oct 31 07:42:54.881000 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880779    4322 cpu_manager.go:414] "containerMap: removing container" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kube-rbac-proxy"
Oct 31 07:42:54.881000 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880784    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kube-rbac-proxy"
Oct 31 07:42:54.881000 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880848    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kserve-container"
Oct 31 07:42:54.881000 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880858    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kube-rbac-proxy"
Oct 31 07:42:54.881000 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880867    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="25bad883-7c4e-459f-a0e8-d14ecde66cca" containerName="kserve-container"
Oct 31 07:42:54.881000 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.880874    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3874750-754d-40dc-8ef1-a214041623a6" containerName="kube-rbac-proxy"
Oct 31 07:42:54.885824 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.885804    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:54.900307 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.900284    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"isvc-sklearn-graph-raw-hpa-529de-kube-rbac-proxy-sar-config\""
Oct 31 07:42:54.900401 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.900346    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"kube-root-ca.crt\""
Oct 31 07:42:54.912167 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.912144    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"openshift-service-ca.crt\""
Oct 31 07:42:54.912308 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.912189    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 07:42:54.912308 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.912211    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"isvc-sklearn-graph-raw-hpa-529de-predictor-serving-cert\""
Oct 31 07:42:54.912308 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.912283    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"seaweedfs-s3-creds\""
Oct 31 07:42:54.912426 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.912315    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"default-dockercfg-d6nzh\""
Oct 31 07:42:54.913892 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.913874    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"]
Oct 31 07:42:54.973870 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.973841    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/672fff04-dde7-4fd3-a4a0-c9371a348303-proxy-tls\") pod \"isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:54.974008 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.973889    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/672fff04-dde7-4fd3-a4a0-c9371a348303-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:54.974008 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.973978    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/672fff04-dde7-4fd3-a4a0-c9371a348303-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:54.974093 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.974025    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/672fff04-dde7-4fd3-a4a0-c9371a348303-cabundle-cert\") pod \"isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:54.974093 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:54.974057    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzqb2\" (UniqueName: \"kubernetes.io/projected/672fff04-dde7-4fd3-a4a0-c9371a348303-kube-api-access-pzqb2\") pod \"isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:55.074666 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.074624    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzqb2\" (UniqueName: \"kubernetes.io/projected/672fff04-dde7-4fd3-a4a0-c9371a348303-kube-api-access-pzqb2\") pod \"isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:55.074805 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.074676    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/672fff04-dde7-4fd3-a4a0-c9371a348303-proxy-tls\") pod \"isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:55.074805 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.074735    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/672fff04-dde7-4fd3-a4a0-c9371a348303-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:55.074805 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.074787    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/672fff04-dde7-4fd3-a4a0-c9371a348303-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:55.074918 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.074838    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/672fff04-dde7-4fd3-a4a0-c9371a348303-cabundle-cert\") pod \"isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:55.075286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.075257    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/672fff04-dde7-4fd3-a4a0-c9371a348303-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:55.075675 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.075652    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/672fff04-dde7-4fd3-a4a0-c9371a348303-cabundle-cert\") pod \"isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:55.075733 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.075652    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/672fff04-dde7-4fd3-a4a0-c9371a348303-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:55.077219 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.077197    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/672fff04-dde7-4fd3-a4a0-c9371a348303-proxy-tls\") pod \"isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:55.097266 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.097246    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzqb2\" (UniqueName: \"kubernetes.io/projected/672fff04-dde7-4fd3-a4a0-c9371a348303-kube-api-access-pzqb2\") pod \"isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:55.202639 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.202562    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:42:55.506392 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.506318    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"]
Oct 31 07:42:55.509966 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:42:55.509930    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod672fff04_dde7_4fd3_a4a0_c9371a348303.slice/crio-3036f878e97fecd923b7e83dbb2d3fc277c1c6e8abe3fcc618e6e9b5515c4489 WatchSource:0}: Error finding container 3036f878e97fecd923b7e83dbb2d3fc277c1c6e8abe3fcc618e6e9b5515c4489: Status 404 returned error can't find the container with id 3036f878e97fecd923b7e83dbb2d3fc277c1c6e8abe3fcc618e6e9b5515c4489
Oct 31 07:42:55.555222 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.555192    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" event={"ID":"672fff04-dde7-4fd3-a4a0-c9371a348303","Type":"ContainerStarted","Data":"3036f878e97fecd923b7e83dbb2d3fc277c1c6e8abe3fcc618e6e9b5515c4489"}
Oct 31 07:42:55.556216 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.556195    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"]
Oct 31 07:42:55.560190 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.560172    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.577395 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.577376    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"isvc-xgboost-graph-raw-hpa-529de-predictor-serving-cert\""
Oct 31 07:42:55.577875 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.577861    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d69f18cd-d869-4c75-a8d6-876d6cd95967-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.577916 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.577889    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d69f18cd-d869-4c75-a8d6-876d6cd95967-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.577960 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.577917    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5nbw\" (UniqueName: \"kubernetes.io/projected/d69f18cd-d869-4c75-a8d6-876d6cd95967-kube-api-access-b5nbw\") pod \"isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.578011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.577973    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d69f18cd-d869-4c75-a8d6-876d6cd95967-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.578123 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.578105    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d69f18cd-d869-4c75-a8d6-876d6cd95967-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.586450 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.586424    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"isvc-xgboost-graph-raw-hpa-529de-kube-rbac-proxy-sar-config\""
Oct 31 07:42:55.587896 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.587876    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"]
Oct 31 07:42:55.679218 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.679180    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d69f18cd-d869-4c75-a8d6-876d6cd95967-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.679419 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.679250    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d69f18cd-d869-4c75-a8d6-876d6cd95967-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.679419 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.679290    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d69f18cd-d869-4c75-a8d6-876d6cd95967-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.679419 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.679341    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5nbw\" (UniqueName: \"kubernetes.io/projected/d69f18cd-d869-4c75-a8d6-876d6cd95967-kube-api-access-b5nbw\") pod \"isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.679419 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.679364    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d69f18cd-d869-4c75-a8d6-876d6cd95967-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.679679 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.679656    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d69f18cd-d869-4c75-a8d6-876d6cd95967-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.679939 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.679919    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d69f18cd-d869-4c75-a8d6-876d6cd95967-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.680041 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.680019    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d69f18cd-d869-4c75-a8d6-876d6cd95967-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.681704 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.681686    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d69f18cd-d869-4c75-a8d6-876d6cd95967-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.701913 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.701882    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5nbw\" (UniqueName: \"kubernetes.io/projected/d69f18cd-d869-4c75-a8d6-876d6cd95967-kube-api-access-b5nbw\") pod \"isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") " pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:55.877176 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:55.877137    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:42:56.193274 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:56.193172    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"]
Oct 31 07:42:56.196538 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:42:56.196509    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd69f18cd_d869_4c75_a8d6_876d6cd95967.slice/crio-6d70de6634e2e0d23511fb3fef1001e99bf04d2cc180267a58f827db71bfdc2a WatchSource:0}: Error finding container 6d70de6634e2e0d23511fb3fef1001e99bf04d2cc180267a58f827db71bfdc2a: Status 404 returned error can't find the container with id 6d70de6634e2e0d23511fb3fef1001e99bf04d2cc180267a58f827db71bfdc2a
Oct 31 07:42:56.561151 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:56.561115    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" event={"ID":"672fff04-dde7-4fd3-a4a0-c9371a348303","Type":"ContainerStarted","Data":"5ecd3a12d344247c7c9168d642ed6de7917989d39da1482fc1f96c2029f216cc"}
Oct 31 07:42:56.562702 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:56.562679    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" event={"ID":"d69f18cd-d869-4c75-a8d6-876d6cd95967","Type":"ContainerStarted","Data":"34bd52601801029a8eba2eaf07dd497fd7c71d8189f4a7eae728338ef9aa9137"}
Oct 31 07:42:56.562841 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:56.562731    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" event={"ID":"d69f18cd-d869-4c75-a8d6-876d6cd95967","Type":"ContainerStarted","Data":"6d70de6634e2e0d23511fb3fef1001e99bf04d2cc180267a58f827db71bfdc2a"}
Oct 31 07:42:59.578537 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:59.578505    4322 generic.go:356] "Generic (PLEG): container finished" podID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerID="5ecd3a12d344247c7c9168d642ed6de7917989d39da1482fc1f96c2029f216cc" exitCode=0
Oct 31 07:42:59.578996 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:42:59.578580    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" event={"ID":"672fff04-dde7-4fd3-a4a0-c9371a348303","Type":"ContainerDied","Data":"5ecd3a12d344247c7c9168d642ed6de7917989d39da1482fc1f96c2029f216cc"}
Oct 31 07:43:00.326787 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:00.326745    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 31 07:43:00.584024 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:00.583939    4322 generic.go:356] "Generic (PLEG): container finished" podID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerID="34bd52601801029a8eba2eaf07dd497fd7c71d8189f4a7eae728338ef9aa9137" exitCode=0
Oct 31 07:43:00.584024 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:00.584008    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" event={"ID":"d69f18cd-d869-4c75-a8d6-876d6cd95967","Type":"ContainerDied","Data":"34bd52601801029a8eba2eaf07dd497fd7c71d8189f4a7eae728338ef9aa9137"}
Oct 31 07:43:00.586415 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:00.586389    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" event={"ID":"672fff04-dde7-4fd3-a4a0-c9371a348303","Type":"ContainerStarted","Data":"f49853633451f361c52fecca329f9e73a86e2231fd90eceff5dd0ae83cdb2446"}
Oct 31 07:43:00.586523 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:00.586440    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" event={"ID":"672fff04-dde7-4fd3-a4a0-c9371a348303","Type":"ContainerStarted","Data":"a791501f4976c66caae3eb5a9110b820739ead8f924eb42cfb14eeb388f7bcea"}
Oct 31 07:43:00.586997 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:00.586981    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:43:00.685611 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:00.685552    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" podStartSLOduration=2.840238202 podStartE2EDuration="6.685535546s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.845297344s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:42:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:43:00.684811316 +0000 UTC m=+821.948631926" watchObservedRunningTime="2026-10-31 07:43:00.685535546 +0000 UTC m=+821.949356158"
Oct 31 07:43:01.593137 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:01.593101    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" event={"ID":"d69f18cd-d869-4c75-a8d6-876d6cd95967","Type":"ContainerStarted","Data":"f8aec42366db320e649d1f18b74134b735a4ab9afcf57b96001dc5adab19f8d9"}
Oct 31 07:43:01.593530 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:01.593169    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" event={"ID":"d69f18cd-d869-4c75-a8d6-876d6cd95967","Type":"ContainerStarted","Data":"929e12e9f49263825e9e47251c7e580eff581c85daf47d49c2ba4150e26baabf"}
Oct 31 07:43:01.593530 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:01.593384    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:43:01.593671 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:01.593652    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:43:01.593859 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:01.593839    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:43:01.594748 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:01.594720    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 31 07:43:01.594834 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:01.594747    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 31 07:43:01.649617 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:01.649569    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" podStartSLOduration=2.929472655 podStartE2EDuration="6.64955219s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.720079535s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:42:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:43:01.64879256 +0000 UTC m=+822.912613170" watchObservedRunningTime="2026-10-31 07:43:01.64955219 +0000 UTC m=+822.913372801"
Oct 31 07:43:01.709880 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:43:01.709844    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:43:02.598259 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:02.598202    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 31 07:43:02.598755 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:02.598313    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 31 07:43:07.604138 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:07.604104    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:43:07.604690 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:07.604670    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:43:07.604769 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:07.604667    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 31 07:43:07.605075 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:07.605051    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 31 07:43:10.326076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:10.326040    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Oct 31 07:43:17.604796 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:17.604759    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 31 07:43:17.605190 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:17.605034    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 31 07:43:20.326889 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:20.326860    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:43:27.605036 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:27.604991    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 31 07:43:27.605036 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:27.605027    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 31 07:43:31.710860 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:43:31.710772    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:43:32.302898 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:32.302868    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"]
Oct 31 07:43:32.303138 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:32.303100    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kserve-container" containerID="cri-o://cff95a48dd124eee6383c46a19c9392c5879c1b0c34a75a99b157cc058bb6138" gracePeriod=30
Oct 31 07:43:32.303319 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:32.303294    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kube-rbac-proxy" containerID="cri-o://b37a9c8fb3374eed9a9262e923900642cc64cfdbfc406374e53c33a3ba622b37" gracePeriod=30
Oct 31 07:43:32.729354 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:32.729279    4322 generic.go:356] "Generic (PLEG): container finished" podID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerID="b37a9c8fb3374eed9a9262e923900642cc64cfdbfc406374e53c33a3ba622b37" exitCode=0
Oct 31 07:43:32.729354 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:32.729337    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" event={"ID":"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb","Type":"ContainerDied","Data":"b37a9c8fb3374eed9a9262e923900642cc64cfdbfc406374e53c33a3ba622b37"}
Oct 31 07:43:35.322382 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:35.322341    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Oct 31 07:43:37.450579 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.450554    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:43:37.452664 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.452644    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-cabundle-cert\") pod \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") "
Oct 31 07:43:37.452759 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.452723    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-proxy-tls\") pod \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") "
Oct 31 07:43:37.452833 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.452805    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kube-rbac-proxy-sar-config\") pod \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") "
Oct 31 07:43:37.452984 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.452965    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-cabundle-cert" pod "f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" (UID: "f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:43:37.453119 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.453100    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kube-rbac-proxy-sar-config" pod "f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" (UID: "f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:43:37.454609 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.454592    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-proxy-tls" pod "f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" (UID: "f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:43:37.553294 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.553269    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kserve-provision-location\") pod \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") "
Oct 31 07:43:37.553449 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.553309    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kube-api-access-7vq9j\" (UniqueName: \"kubernetes.io/projected/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kube-api-access-7vq9j\") pod \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\" (UID: \"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb\") "
Oct 31 07:43:37.553575 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.553552    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kserve-provision-location" pod "f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" (UID: "f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:43:37.553575 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.553569    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:43:37.553673 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.553589    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:43:37.553673 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.553605    4322 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-cabundle-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:43:37.555194 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.555174    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kube-api-access-7vq9j" pod "f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" (UID: "f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb"). InnerVolumeSpecName "kube-api-access-7vq9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:43:37.605352 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.605324    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 31 07:43:37.605459 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.605324    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 31 07:43:37.653994 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.653973    4322 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kserve-provision-location\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:43:37.653994 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.653995    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vq9j\" (UniqueName: \"kubernetes.io/projected/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb-kube-api-access-7vq9j\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:43:37.754111 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.754084    4322 generic.go:356] "Generic (PLEG): container finished" podID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerID="cff95a48dd124eee6383c46a19c9392c5879c1b0c34a75a99b157cc058bb6138" exitCode=0
Oct 31 07:43:37.754260 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.754138    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"
Oct 31 07:43:37.754260 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.754164    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" event={"ID":"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb","Type":"ContainerDied","Data":"cff95a48dd124eee6383c46a19c9392c5879c1b0c34a75a99b157cc058bb6138"}
Oct 31 07:43:37.754260 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.754245    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c" event={"ID":"f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb","Type":"ContainerDied","Data":"f99a5a3a68d259a27eea4170f1cf618e558594807bb0224283f5c8efe59ebfd7"}
Oct 31 07:43:37.754260 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.754256    4322 scope.go:122] "RemoveContainer" containerID="b37a9c8fb3374eed9a9262e923900642cc64cfdbfc406374e53c33a3ba622b37"
Oct 31 07:43:37.762645 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.762626    4322 scope.go:122] "RemoveContainer" containerID="cff95a48dd124eee6383c46a19c9392c5879c1b0c34a75a99b157cc058bb6138"
Oct 31 07:43:37.775636 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.775604    4322 scope.go:122] "RemoveContainer" containerID="7c7bab6aac0b1a266c6ebc5b0d649ca4a88f04cfd1ba7d547eb3f1ab3eb03a9f"
Oct 31 07:43:37.783267 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.783242    4322 scope.go:122] "RemoveContainer" containerID="b37a9c8fb3374eed9a9262e923900642cc64cfdbfc406374e53c33a3ba622b37"
Oct 31 07:43:37.783535 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:43:37.783517    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b37a9c8fb3374eed9a9262e923900642cc64cfdbfc406374e53c33a3ba622b37\": container with ID starting with b37a9c8fb3374eed9a9262e923900642cc64cfdbfc406374e53c33a3ba622b37 not found: ID does not exist" containerID="b37a9c8fb3374eed9a9262e923900642cc64cfdbfc406374e53c33a3ba622b37"
Oct 31 07:43:37.783581 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.783544    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b37a9c8fb3374eed9a9262e923900642cc64cfdbfc406374e53c33a3ba622b37"} err="failed to get container status \"b37a9c8fb3374eed9a9262e923900642cc64cfdbfc406374e53c33a3ba622b37\": rpc error: code = NotFound desc = could not find container \"b37a9c8fb3374eed9a9262e923900642cc64cfdbfc406374e53c33a3ba622b37\": container with ID starting with b37a9c8fb3374eed9a9262e923900642cc64cfdbfc406374e53c33a3ba622b37 not found: ID does not exist"
Oct 31 07:43:37.783581 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.783561    4322 scope.go:122] "RemoveContainer" containerID="cff95a48dd124eee6383c46a19c9392c5879c1b0c34a75a99b157cc058bb6138"
Oct 31 07:43:37.783793 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:43:37.783772    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cff95a48dd124eee6383c46a19c9392c5879c1b0c34a75a99b157cc058bb6138\": container with ID starting with cff95a48dd124eee6383c46a19c9392c5879c1b0c34a75a99b157cc058bb6138 not found: ID does not exist" containerID="cff95a48dd124eee6383c46a19c9392c5879c1b0c34a75a99b157cc058bb6138"
Oct 31 07:43:37.783837 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.783800    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cff95a48dd124eee6383c46a19c9392c5879c1b0c34a75a99b157cc058bb6138"} err="failed to get container status \"cff95a48dd124eee6383c46a19c9392c5879c1b0c34a75a99b157cc058bb6138\": rpc error: code = NotFound desc = could not find container \"cff95a48dd124eee6383c46a19c9392c5879c1b0c34a75a99b157cc058bb6138\": container with ID starting with cff95a48dd124eee6383c46a19c9392c5879c1b0c34a75a99b157cc058bb6138 not found: ID does not exist"
Oct 31 07:43:37.783837 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.783815    4322 scope.go:122] "RemoveContainer" containerID="7c7bab6aac0b1a266c6ebc5b0d649ca4a88f04cfd1ba7d547eb3f1ab3eb03a9f"
Oct 31 07:43:37.784018 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:43:37.784000    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c7bab6aac0b1a266c6ebc5b0d649ca4a88f04cfd1ba7d547eb3f1ab3eb03a9f\": container with ID starting with 7c7bab6aac0b1a266c6ebc5b0d649ca4a88f04cfd1ba7d547eb3f1ab3eb03a9f not found: ID does not exist" containerID="7c7bab6aac0b1a266c6ebc5b0d649ca4a88f04cfd1ba7d547eb3f1ab3eb03a9f"
Oct 31 07:43:37.784077 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.784025    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c7bab6aac0b1a266c6ebc5b0d649ca4a88f04cfd1ba7d547eb3f1ab3eb03a9f"} err="failed to get container status \"7c7bab6aac0b1a266c6ebc5b0d649ca4a88f04cfd1ba7d547eb3f1ab3eb03a9f\": rpc error: code = NotFound desc = could not find container \"7c7bab6aac0b1a266c6ebc5b0d649ca4a88f04cfd1ba7d547eb3f1ab3eb03a9f\": container with ID starting with 7c7bab6aac0b1a266c6ebc5b0d649ca4a88f04cfd1ba7d547eb3f1ab3eb03a9f not found: ID does not exist"
Oct 31 07:43:37.806569 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.806505    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"]
Oct 31 07:43:37.809911 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:37.809888    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-stop-predictor-589d977cbd-fq75c"]
Oct 31 07:43:38.871328 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:38.871297    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" path="/var/lib/kubelet/pods/f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb/volumes"
Oct 31 07:43:47.605220 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:47.605184    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 31 07:43:47.605220 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:47.605184    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 31 07:43:57.605607 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:57.605566    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 31 07:43:57.606043 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:43:57.605569    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 31 07:44:01.709557 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:44:01.709518    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:44:07.605595 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:07.605554    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 31 07:44:07.606055 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:07.605554    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 31 07:44:14.866087 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:14.866042    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 31 07:44:17.606125 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:17.606091    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:44:24.871174 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:24.871142    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:44:31.711090 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:44:31.711054    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:44:55.463862 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:55.463827    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"]
Oct 31 07:44:55.464437 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:55.464137    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kserve-container" containerID="cri-o://a791501f4976c66caae3eb5a9110b820739ead8f924eb42cfb14eeb388f7bcea" gracePeriod=30
Oct 31 07:44:55.464437 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:55.464260    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kube-rbac-proxy" containerID="cri-o://f49853633451f361c52fecca329f9e73a86e2231fd90eceff5dd0ae83cdb2446" gracePeriod=30
Oct 31 07:44:55.550713 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:55.550684    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"]
Oct 31 07:44:55.551116 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:55.550932    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kserve-container" containerID="cri-o://929e12e9f49263825e9e47251c7e580eff581c85daf47d49c2ba4150e26baabf" gracePeriod=30
Oct 31 07:44:55.551116 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:55.550984    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kube-rbac-proxy" containerID="cri-o://f8aec42366db320e649d1f18b74134b735a4ab9afcf57b96001dc5adab19f8d9" gracePeriod=30
Oct 31 07:44:56.087628 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:56.087596    4322 generic.go:356] "Generic (PLEG): container finished" podID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerID="f8aec42366db320e649d1f18b74134b735a4ab9afcf57b96001dc5adab19f8d9" exitCode=0
Oct 31 07:44:56.087824 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:56.087667    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" event={"ID":"d69f18cd-d869-4c75-a8d6-876d6cd95967","Type":"ContainerDied","Data":"f8aec42366db320e649d1f18b74134b735a4ab9afcf57b96001dc5adab19f8d9"}
Oct 31 07:44:56.089664 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:56.089644    4322 generic.go:356] "Generic (PLEG): container finished" podID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerID="f49853633451f361c52fecca329f9e73a86e2231fd90eceff5dd0ae83cdb2446" exitCode=0
Oct 31 07:44:56.089751 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:56.089699    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" event={"ID":"672fff04-dde7-4fd3-a4a0-c9371a348303","Type":"ContainerDied","Data":"f49853633451f361c52fecca329f9e73a86e2231fd90eceff5dd0ae83cdb2446"}
Oct 31 07:44:57.598552 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:57.598513    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Oct 31 07:44:57.598944 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:57.598515    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.35:8643/healthz\": dial tcp 10.134.0.35:8643: connect: connection refused"
Oct 31 07:44:57.605903 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:57.605881    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Oct 31 07:44:59.594594 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.594567    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:44:59.757538 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.757489    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d69f18cd-d869-4c75-a8d6-876d6cd95967-proxy-tls\" (UniqueName: \"kubernetes.io/secret/d69f18cd-d869-4c75-a8d6-876d6cd95967-proxy-tls\") pod \"d69f18cd-d869-4c75-a8d6-876d6cd95967\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") "
Oct 31 07:44:59.757538 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.757522    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d69f18cd-d869-4c75-a8d6-876d6cd95967-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d69f18cd-d869-4c75-a8d6-876d6cd95967-kserve-provision-location\") pod \"d69f18cd-d869-4c75-a8d6-876d6cd95967\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") "
Oct 31 07:44:59.757718 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.757567    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d69f18cd-d869-4c75-a8d6-876d6cd95967-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d69f18cd-d869-4c75-a8d6-876d6cd95967-cabundle-cert\") pod \"d69f18cd-d869-4c75-a8d6-876d6cd95967\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") "
Oct 31 07:44:59.757718 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.757616    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d69f18cd-d869-4c75-a8d6-876d6cd95967-kube-api-access-b5nbw\" (UniqueName: \"kubernetes.io/projected/d69f18cd-d869-4c75-a8d6-876d6cd95967-kube-api-access-b5nbw\") pod \"d69f18cd-d869-4c75-a8d6-876d6cd95967\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") "
Oct 31 07:44:59.757718 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.757682    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d69f18cd-d869-4c75-a8d6-876d6cd95967-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d69f18cd-d869-4c75-a8d6-876d6cd95967-kube-rbac-proxy-sar-config\") pod \"d69f18cd-d869-4c75-a8d6-876d6cd95967\" (UID: \"d69f18cd-d869-4c75-a8d6-876d6cd95967\") "
Oct 31 07:44:59.757891 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.757861    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d69f18cd-d869-4c75-a8d6-876d6cd95967-kserve-provision-location" pod "d69f18cd-d869-4c75-a8d6-876d6cd95967" (UID: "d69f18cd-d869-4c75-a8d6-876d6cd95967"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:44:59.757958 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.757934    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d69f18cd-d869-4c75-a8d6-876d6cd95967-cabundle-cert" pod "d69f18cd-d869-4c75-a8d6-876d6cd95967" (UID: "d69f18cd-d869-4c75-a8d6-876d6cd95967"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:44:59.758132 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.758108    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d69f18cd-d869-4c75-a8d6-876d6cd95967-kube-rbac-proxy-sar-config" pod "d69f18cd-d869-4c75-a8d6-876d6cd95967" (UID: "d69f18cd-d869-4c75-a8d6-876d6cd95967"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:44:59.759681 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.759657    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d69f18cd-d869-4c75-a8d6-876d6cd95967-kube-api-access-b5nbw" pod "d69f18cd-d869-4c75-a8d6-876d6cd95967" (UID: "d69f18cd-d869-4c75-a8d6-876d6cd95967"). InnerVolumeSpecName "kube-api-access-b5nbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:44:59.759739 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.759686    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d69f18cd-d869-4c75-a8d6-876d6cd95967-proxy-tls" pod "d69f18cd-d869-4c75-a8d6-876d6cd95967" (UID: "d69f18cd-d869-4c75-a8d6-876d6cd95967"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:44:59.859255 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.859217    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5nbw\" (UniqueName: \"kubernetes.io/projected/d69f18cd-d869-4c75-a8d6-876d6cd95967-kube-api-access-b5nbw\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:44:59.859255 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.859254    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d69f18cd-d869-4c75-a8d6-876d6cd95967-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:44:59.859424 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.859265    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d69f18cd-d869-4c75-a8d6-876d6cd95967-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:44:59.859424 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.859275    4322 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d69f18cd-d869-4c75-a8d6-876d6cd95967-kserve-provision-location\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:44:59.859424 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:44:59.859285    4322 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d69f18cd-d869-4c75-a8d6-876d6cd95967-cabundle-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:45:00.108482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.108445    4322 generic.go:356] "Generic (PLEG): container finished" podID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerID="929e12e9f49263825e9e47251c7e580eff581c85daf47d49c2ba4150e26baabf" exitCode=0
Oct 31 07:45:00.108659 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.108515    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"
Oct 31 07:45:00.108659 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.108531    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" event={"ID":"d69f18cd-d869-4c75-a8d6-876d6cd95967","Type":"ContainerDied","Data":"929e12e9f49263825e9e47251c7e580eff581c85daf47d49c2ba4150e26baabf"}
Oct 31 07:45:00.108659 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.108590    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv" event={"ID":"d69f18cd-d869-4c75-a8d6-876d6cd95967","Type":"ContainerDied","Data":"6d70de6634e2e0d23511fb3fef1001e99bf04d2cc180267a58f827db71bfdc2a"}
Oct 31 07:45:00.108659 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.108617    4322 scope.go:122] "RemoveContainer" containerID="f8aec42366db320e649d1f18b74134b735a4ab9afcf57b96001dc5adab19f8d9"
Oct 31 07:45:00.112720 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.112694    4322 generic.go:356] "Generic (PLEG): container finished" podID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerID="a791501f4976c66caae3eb5a9110b820739ead8f924eb42cfb14eeb388f7bcea" exitCode=0
Oct 31 07:45:00.112843 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.112766    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" event={"ID":"672fff04-dde7-4fd3-a4a0-c9371a348303","Type":"ContainerDied","Data":"a791501f4976c66caae3eb5a9110b820739ead8f924eb42cfb14eeb388f7bcea"}
Oct 31 07:45:00.119435 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.119413    4322 scope.go:122] "RemoveContainer" containerID="929e12e9f49263825e9e47251c7e580eff581c85daf47d49c2ba4150e26baabf"
Oct 31 07:45:00.127512 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.127496    4322 scope.go:122] "RemoveContainer" containerID="34bd52601801029a8eba2eaf07dd497fd7c71d8189f4a7eae728338ef9aa9137"
Oct 31 07:45:00.135068 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.135051    4322 scope.go:122] "RemoveContainer" containerID="f8aec42366db320e649d1f18b74134b735a4ab9afcf57b96001dc5adab19f8d9"
Oct 31 07:45:00.135402 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:45:00.135374    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8aec42366db320e649d1f18b74134b735a4ab9afcf57b96001dc5adab19f8d9\": container with ID starting with f8aec42366db320e649d1f18b74134b735a4ab9afcf57b96001dc5adab19f8d9 not found: ID does not exist" containerID="f8aec42366db320e649d1f18b74134b735a4ab9afcf57b96001dc5adab19f8d9"
Oct 31 07:45:00.135521 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.135411    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8aec42366db320e649d1f18b74134b735a4ab9afcf57b96001dc5adab19f8d9"} err="failed to get container status \"f8aec42366db320e649d1f18b74134b735a4ab9afcf57b96001dc5adab19f8d9\": rpc error: code = NotFound desc = could not find container \"f8aec42366db320e649d1f18b74134b735a4ab9afcf57b96001dc5adab19f8d9\": container with ID starting with f8aec42366db320e649d1f18b74134b735a4ab9afcf57b96001dc5adab19f8d9 not found: ID does not exist"
Oct 31 07:45:00.135521 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.135438    4322 scope.go:122] "RemoveContainer" containerID="929e12e9f49263825e9e47251c7e580eff581c85daf47d49c2ba4150e26baabf"
Oct 31 07:45:00.135721 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:45:00.135700    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"929e12e9f49263825e9e47251c7e580eff581c85daf47d49c2ba4150e26baabf\": container with ID starting with 929e12e9f49263825e9e47251c7e580eff581c85daf47d49c2ba4150e26baabf not found: ID does not exist" containerID="929e12e9f49263825e9e47251c7e580eff581c85daf47d49c2ba4150e26baabf"
Oct 31 07:45:00.135785 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.135732    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"929e12e9f49263825e9e47251c7e580eff581c85daf47d49c2ba4150e26baabf"} err="failed to get container status \"929e12e9f49263825e9e47251c7e580eff581c85daf47d49c2ba4150e26baabf\": rpc error: code = NotFound desc = could not find container \"929e12e9f49263825e9e47251c7e580eff581c85daf47d49c2ba4150e26baabf\": container with ID starting with 929e12e9f49263825e9e47251c7e580eff581c85daf47d49c2ba4150e26baabf not found: ID does not exist"
Oct 31 07:45:00.135785 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.135750    4322 scope.go:122] "RemoveContainer" containerID="34bd52601801029a8eba2eaf07dd497fd7c71d8189f4a7eae728338ef9aa9137"
Oct 31 07:45:00.136040 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:45:00.136012    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34bd52601801029a8eba2eaf07dd497fd7c71d8189f4a7eae728338ef9aa9137\": container with ID starting with 34bd52601801029a8eba2eaf07dd497fd7c71d8189f4a7eae728338ef9aa9137 not found: ID does not exist" containerID="34bd52601801029a8eba2eaf07dd497fd7c71d8189f4a7eae728338ef9aa9137"
Oct 31 07:45:00.136134 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.136048    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34bd52601801029a8eba2eaf07dd497fd7c71d8189f4a7eae728338ef9aa9137"} err="failed to get container status \"34bd52601801029a8eba2eaf07dd497fd7c71d8189f4a7eae728338ef9aa9137\": rpc error: code = NotFound desc = could not find container \"34bd52601801029a8eba2eaf07dd497fd7c71d8189f4a7eae728338ef9aa9137\": container with ID starting with 34bd52601801029a8eba2eaf07dd497fd7c71d8189f4a7eae728338ef9aa9137 not found: ID does not exist"
Oct 31 07:45:00.171623 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.171589    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"]
Oct 31 07:45:00.184311 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.184280    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-xgboost-graph-raw-hpa-529de-predictor-55ccb8d7c-6zsxv"]
Oct 31 07:45:00.208579 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.208556    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:45:00.364587 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.364509    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/672fff04-dde7-4fd3-a4a0-c9371a348303-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/672fff04-dde7-4fd3-a4a0-c9371a348303-kserve-provision-location\") pod \"672fff04-dde7-4fd3-a4a0-c9371a348303\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") "
Oct 31 07:45:00.364587 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.364564    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/672fff04-dde7-4fd3-a4a0-c9371a348303-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/672fff04-dde7-4fd3-a4a0-c9371a348303-kube-rbac-proxy-sar-config\") pod \"672fff04-dde7-4fd3-a4a0-c9371a348303\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") "
Oct 31 07:45:00.364825 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.364594    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/672fff04-dde7-4fd3-a4a0-c9371a348303-kube-api-access-pzqb2\" (UniqueName: \"kubernetes.io/projected/672fff04-dde7-4fd3-a4a0-c9371a348303-kube-api-access-pzqb2\") pod \"672fff04-dde7-4fd3-a4a0-c9371a348303\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") "
Oct 31 07:45:00.364825 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.364621    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/672fff04-dde7-4fd3-a4a0-c9371a348303-proxy-tls\" (UniqueName: \"kubernetes.io/secret/672fff04-dde7-4fd3-a4a0-c9371a348303-proxy-tls\") pod \"672fff04-dde7-4fd3-a4a0-c9371a348303\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") "
Oct 31 07:45:00.364825 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.364703    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/672fff04-dde7-4fd3-a4a0-c9371a348303-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/672fff04-dde7-4fd3-a4a0-c9371a348303-cabundle-cert\") pod \"672fff04-dde7-4fd3-a4a0-c9371a348303\" (UID: \"672fff04-dde7-4fd3-a4a0-c9371a348303\") "
Oct 31 07:45:00.364976 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.364842    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/672fff04-dde7-4fd3-a4a0-c9371a348303-kserve-provision-location" pod "672fff04-dde7-4fd3-a4a0-c9371a348303" (UID: "672fff04-dde7-4fd3-a4a0-c9371a348303"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:45:00.365059 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.365033    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/672fff04-dde7-4fd3-a4a0-c9371a348303-kube-rbac-proxy-sar-config" pod "672fff04-dde7-4fd3-a4a0-c9371a348303" (UID: "672fff04-dde7-4fd3-a4a0-c9371a348303"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:45:00.365116 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.365102    4322 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/672fff04-dde7-4fd3-a4a0-c9371a348303-kserve-provision-location\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:45:00.365156 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.365111    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/672fff04-dde7-4fd3-a4a0-c9371a348303-cabundle-cert" pod "672fff04-dde7-4fd3-a4a0-c9371a348303" (UID: "672fff04-dde7-4fd3-a4a0-c9371a348303"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:45:00.366691 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.366672    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/672fff04-dde7-4fd3-a4a0-c9371a348303-kube-api-access-pzqb2" pod "672fff04-dde7-4fd3-a4a0-c9371a348303" (UID: "672fff04-dde7-4fd3-a4a0-c9371a348303"). InnerVolumeSpecName "kube-api-access-pzqb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:45:00.366691 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.366684    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/672fff04-dde7-4fd3-a4a0-c9371a348303-proxy-tls" pod "672fff04-dde7-4fd3-a4a0-c9371a348303" (UID: "672fff04-dde7-4fd3-a4a0-c9371a348303"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:45:00.466380 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.466354    4322 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/672fff04-dde7-4fd3-a4a0-c9371a348303-cabundle-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:45:00.466380 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.466380    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/672fff04-dde7-4fd3-a4a0-c9371a348303-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:45:00.466533 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.466390    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzqb2\" (UniqueName: \"kubernetes.io/projected/672fff04-dde7-4fd3-a4a0-c9371a348303-kube-api-access-pzqb2\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:45:00.466533 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.466400    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/672fff04-dde7-4fd3-a4a0-c9371a348303-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:45:00.870222 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:00.870133    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" path="/var/lib/kubelet/pods/d69f18cd-d869-4c75-a8d6-876d6cd95967/volumes"
Oct 31 07:45:01.119910 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:01.119880    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q" event={"ID":"672fff04-dde7-4fd3-a4a0-c9371a348303","Type":"ContainerDied","Data":"3036f878e97fecd923b7e83dbb2d3fc277c1c6e8abe3fcc618e6e9b5515c4489"}
Oct 31 07:45:01.120078 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:01.119922    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"
Oct 31 07:45:01.120078 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:01.119944    4322 scope.go:122] "RemoveContainer" containerID="f49853633451f361c52fecca329f9e73a86e2231fd90eceff5dd0ae83cdb2446"
Oct 31 07:45:01.127965 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:01.127944    4322 scope.go:122] "RemoveContainer" containerID="a791501f4976c66caae3eb5a9110b820739ead8f924eb42cfb14eeb388f7bcea"
Oct 31 07:45:01.135189 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:01.135171    4322 scope.go:122] "RemoveContainer" containerID="5ecd3a12d344247c7c9168d642ed6de7917989d39da1482fc1f96c2029f216cc"
Oct 31 07:45:01.185867 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:01.185840    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"]
Oct 31 07:45:01.189713 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:01.189688    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-graph-raw-hpa-529de-predictor-6d4bfcb897-brk7q"]
Oct 31 07:45:01.710898 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:45:01.710864    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:45:02.872001 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.871964    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" path="/var/lib/kubelet/pods/672fff04-dde7-4fd3-a4a0-c9371a348303/volumes"
Oct 31 07:45:02.978118 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.978085    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"]
Oct 31 07:45:02.979102 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979074    4322 cpu_manager.go:414] "containerMap: removing container" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="storage-initializer"
Oct 31 07:45:02.979102 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979101    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="storage-initializer"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979131    4322 cpu_manager.go:414] "containerMap: removing container" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="storage-initializer"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979140    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="storage-initializer"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979153    4322 cpu_manager.go:414] "containerMap: removing container" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kube-rbac-proxy"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979161    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kube-rbac-proxy"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979185    4322 cpu_manager.go:414] "containerMap: removing container" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kserve-container"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979194    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kserve-container"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979296    4322 cpu_manager.go:414] "containerMap: removing container" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kube-rbac-proxy"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979303    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kube-rbac-proxy"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979329    4322 cpu_manager.go:414] "containerMap: removing container" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="storage-initializer"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979337    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="storage-initializer"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979349    4322 cpu_manager.go:414] "containerMap: removing container" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kserve-container"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979357    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kserve-container"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979381    4322 cpu_manager.go:414] "containerMap: removing container" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kserve-container"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979387    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kserve-container"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979470    4322 cpu_manager.go:414] "containerMap: removing container" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kube-rbac-proxy"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979483    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kube-rbac-proxy"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.979988    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kube-rbac-proxy"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.980009    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kserve-container"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.980021    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="d69f18cd-d869-4c75-a8d6-876d6cd95967" containerName="kube-rbac-proxy"
Oct 31 07:45:02.980020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.980031    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kserve-container"
Oct 31 07:45:02.980953 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.980047    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="672fff04-dde7-4fd3-a4a0-c9371a348303" containerName="kserve-container"
Oct 31 07:45:02.980953 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.980092    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6ea81a4-8199-49cc-bca8-8dfa6b2dbccb" containerName="kube-rbac-proxy"
Oct 31 07:45:02.989721 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:02.989694    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:45:03.002144 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.002119    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"]
Oct 31 07:45:03.012939 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.012912    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"message-dumper-raw-6ca8d-kube-rbac-proxy-sar-config\""
Oct 31 07:45:03.013063 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.012960    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"openshift-service-ca.crt\""
Oct 31 07:45:03.013063 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.012968    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"message-dumper-raw-6ca8d-predictor-serving-cert\""
Oct 31 07:45:03.013063 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.012970    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"kube-root-ca.crt\""
Oct 31 07:45:03.013183 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.013073    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"default-dockercfg-d6nzh\""
Oct 31 07:45:03.088051 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.088030    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c4f1cbd9-37df-4c8b-b162-e73acd77a326-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9\" (UID: \"c4f1cbd9-37df-4c8b-b162-e73acd77a326\") " pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:45:03.088161 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.088068    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpl4v\" (UniqueName: \"kubernetes.io/projected/c4f1cbd9-37df-4c8b-b162-e73acd77a326-kube-api-access-zpl4v\") pod \"message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9\" (UID: \"c4f1cbd9-37df-4c8b-b162-e73acd77a326\") " pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:45:03.088161 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.088088    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c4f1cbd9-37df-4c8b-b162-e73acd77a326-proxy-tls\") pod \"message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9\" (UID: \"c4f1cbd9-37df-4c8b-b162-e73acd77a326\") " pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:45:03.188873 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.188811    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c4f1cbd9-37df-4c8b-b162-e73acd77a326-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9\" (UID: \"c4f1cbd9-37df-4c8b-b162-e73acd77a326\") " pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:45:03.188873 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.188850    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpl4v\" (UniqueName: \"kubernetes.io/projected/c4f1cbd9-37df-4c8b-b162-e73acd77a326-kube-api-access-zpl4v\") pod \"message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9\" (UID: \"c4f1cbd9-37df-4c8b-b162-e73acd77a326\") " pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:45:03.189026 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.188872    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c4f1cbd9-37df-4c8b-b162-e73acd77a326-proxy-tls\") pod \"message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9\" (UID: \"c4f1cbd9-37df-4c8b-b162-e73acd77a326\") " pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:45:03.189447 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.189421    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c4f1cbd9-37df-4c8b-b162-e73acd77a326-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9\" (UID: \"c4f1cbd9-37df-4c8b-b162-e73acd77a326\") " pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:45:03.191421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.191401    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c4f1cbd9-37df-4c8b-b162-e73acd77a326-proxy-tls\") pod \"message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9\" (UID: \"c4f1cbd9-37df-4c8b-b162-e73acd77a326\") " pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:45:03.211830 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.211804    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpl4v\" (UniqueName: \"kubernetes.io/projected/c4f1cbd9-37df-4c8b-b162-e73acd77a326-kube-api-access-zpl4v\") pod \"message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9\" (UID: \"c4f1cbd9-37df-4c8b-b162-e73acd77a326\") " pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:45:03.304763 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.304735    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:45:03.594922 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:03.594897    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"]
Oct 31 07:45:03.597932 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:45:03.597908    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4f1cbd9_37df_4c8b_b162_e73acd77a326.slice/crio-85c9bca8321e955c93f5cd4a6cffb9632cd07cea95baafa513e0cf64f4b6d90b WatchSource:0}: Error finding container 85c9bca8321e955c93f5cd4a6cffb9632cd07cea95baafa513e0cf64f4b6d90b: Status 404 returned error can't find the container with id 85c9bca8321e955c93f5cd4a6cffb9632cd07cea95baafa513e0cf64f4b6d90b
Oct 31 07:45:04.134765 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:04.134731    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9" event={"ID":"c4f1cbd9-37df-4c8b-b162-e73acd77a326","Type":"ContainerStarted","Data":"85c9bca8321e955c93f5cd4a6cffb9632cd07cea95baafa513e0cf64f4b6d90b"}
Oct 31 07:45:05.140497 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:05.140404    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9" event={"ID":"c4f1cbd9-37df-4c8b-b162-e73acd77a326","Type":"ContainerStarted","Data":"d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7"}
Oct 31 07:45:05.140497 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:05.140463    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9" event={"ID":"c4f1cbd9-37df-4c8b-b162-e73acd77a326","Type":"ContainerStarted","Data":"d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9"}
Oct 31 07:45:05.140890 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:05.140500    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:45:05.182611 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:05.182565    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9" podStartSLOduration=1.93034096 podStartE2EDuration="3.182553648s" totalImagesPullingTime="1.252212688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:45:05.181436982 +0000 UTC m=+946.445257593" watchObservedRunningTime="2026-10-31 07:45:05.182553648 +0000 UTC m=+946.446374258"
Oct 31 07:45:06.144432 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:06.144400    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:45:06.146367 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:06.146345    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:45:13.159334 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:13.159304    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:45:23.153120 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.153086    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"]
Oct 31 07:45:23.158530 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.158509    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.172970 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.172946    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"seaweedfs-s3-creds\""
Oct 31 07:45:23.173074 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.172992    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"isvc-logger-raw-6ca8d-kube-rbac-proxy-sar-config\""
Oct 31 07:45:23.173074 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.173003    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 07:45:23.184723 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.184706    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"isvc-logger-raw-6ca8d-predictor-serving-cert\""
Oct 31 07:45:23.187080 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.187061    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"]
Oct 31 07:45:23.250336 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.250306    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdzx5\" (UniqueName: \"kubernetes.io/projected/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kube-api-access-mdzx5\") pod \"isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") " pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.250450 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.250361    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kserve-provision-location\") pod \"isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") " pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.250450 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.250382    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a8675074-6daf-4c11-9c2b-2ed73c7900b0-cabundle-cert\") pod \"isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") " pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.250560 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.250453    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") " pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.250560 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.250539    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a8675074-6daf-4c11-9c2b-2ed73c7900b0-proxy-tls\") pod \"isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") " pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.351882 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.351852    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a8675074-6daf-4c11-9c2b-2ed73c7900b0-proxy-tls\") pod \"isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") " pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.351994 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.351888    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdzx5\" (UniqueName: \"kubernetes.io/projected/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kube-api-access-mdzx5\") pod \"isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") " pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.351994 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.351921    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kserve-provision-location\") pod \"isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") " pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.351994 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.351936    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a8675074-6daf-4c11-9c2b-2ed73c7900b0-cabundle-cert\") pod \"isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") " pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.351994 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.351956    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") " pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.352335 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.352316    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kserve-provision-location\") pod \"isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") " pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.352663 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.352645    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a8675074-6daf-4c11-9c2b-2ed73c7900b0-cabundle-cert\") pod \"isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") " pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.352738 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.352669    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") " pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.354591 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.354567    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a8675074-6daf-4c11-9c2b-2ed73c7900b0-proxy-tls\") pod \"isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") " pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.374934 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.374912    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdzx5\" (UniqueName: \"kubernetes.io/projected/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kube-api-access-mdzx5\") pod \"isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") " pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.477237 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.477132    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:23.799565 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:23.799528    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"]
Oct 31 07:45:23.803897 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:45:23.803868    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8675074_6daf_4c11_9c2b_2ed73c7900b0.slice/crio-23fdd8e92ba28a6bcae0a6093f653f29c5f58eb084b4b4c78c33f9197e273719 WatchSource:0}: Error finding container 23fdd8e92ba28a6bcae0a6093f653f29c5f58eb084b4b4c78c33f9197e273719: Status 404 returned error can't find the container with id 23fdd8e92ba28a6bcae0a6093f653f29c5f58eb084b4b4c78c33f9197e273719
Oct 31 07:45:24.227590 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:24.227509    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" event={"ID":"a8675074-6daf-4c11-9c2b-2ed73c7900b0","Type":"ContainerStarted","Data":"f27765ad68c109a49468128b2de6dd8ab485040ec823a0cae2d5f697cd58b6cd"}
Oct 31 07:45:24.227590 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:24.227577    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" event={"ID":"a8675074-6daf-4c11-9c2b-2ed73c7900b0","Type":"ContainerStarted","Data":"23fdd8e92ba28a6bcae0a6093f653f29c5f58eb084b4b4c78c33f9197e273719"}
Oct 31 07:45:28.246279 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:28.246213    4322 generic.go:356] "Generic (PLEG): container finished" podID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerID="f27765ad68c109a49468128b2de6dd8ab485040ec823a0cae2d5f697cd58b6cd" exitCode=0
Oct 31 07:45:28.246648 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:28.246293    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" event={"ID":"a8675074-6daf-4c11-9c2b-2ed73c7900b0","Type":"ContainerDied","Data":"f27765ad68c109a49468128b2de6dd8ab485040ec823a0cae2d5f697cd58b6cd"}
Oct 31 07:45:29.254510 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:29.254478    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" event={"ID":"a8675074-6daf-4c11-9c2b-2ed73c7900b0","Type":"ContainerStarted","Data":"c0fd344e525b200e55716c3e9ae3fc8ba2998c1678e601444ad70c82f2cbcdaf"}
Oct 31 07:45:29.254897 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:29.254571    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" event={"ID":"a8675074-6daf-4c11-9c2b-2ed73c7900b0","Type":"ContainerStarted","Data":"9b11252e2155b452d3b77061412e1e1f584c94cbd1a3696572ce8b1ae0be31d8"}
Oct 31 07:45:29.254897 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:29.254606    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" event={"ID":"a8675074-6daf-4c11-9c2b-2ed73c7900b0","Type":"ContainerStarted","Data":"049c4736b4b85e17c2cda75d6678f7481645b11f6dcecd88bee67e8c9cb842fb"}
Oct 31 07:45:29.254897 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:29.254854    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:29.255086 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:29.255010    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:29.256221 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:29.256192    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Oct 31 07:45:29.322320 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:29.322275    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podStartSLOduration=2.6570080259999997 podStartE2EDuration="6.322262721s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.665254695s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:45:29.309116749 +0000 UTC m=+970.572937498" watchObservedRunningTime="2026-10-31 07:45:29.322262721 +0000 UTC m=+970.586083333"
Oct 31 07:45:30.259757 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:30.259713    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:30.260257 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:30.259855    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Oct 31 07:45:30.261047 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:30.261018    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:45:31.264579 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:31.264535    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Oct 31 07:45:31.265083 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:31.264894    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:45:31.709729 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:45:31.709644    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:45:36.269097 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:36.269064    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:45:36.269805 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:36.269694    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Oct 31 07:45:36.269993 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:36.269968    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:45:46.270588 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:46.270540    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Oct 31 07:45:46.271010 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:46.270915    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:45:56.270397 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:56.270353    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Oct 31 07:45:56.270866 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:45:56.270842    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:46:01.710871 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:46:01.710827    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:46:06.269601 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:06.269566    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Oct 31 07:46:06.270072 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:06.270029    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:46:16.269920 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:16.269878    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Oct 31 07:46:16.270409 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:16.270387    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:46:26.270568 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:26.270524    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Oct 31 07:46:26.271075 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:26.271048    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:46:31.710437 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:46:31.710358    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:46:36.269880 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:36.269847    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Oct 31 07:46:36.270372 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:36.270347    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:46:46.270148 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:46.270106    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:46:46.270636 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:46.270326    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:46:58.610222 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:58.610189    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-7f5ae2e6e4_message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9_c4f1cbd9-37df-4c8b-b162-e73acd77a326/kserve-container/0.log"
Oct 31 07:46:59.163241 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.163194    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"]
Oct 31 07:46:59.163575 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.163525    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container" containerID="cri-o://049c4736b4b85e17c2cda75d6678f7481645b11f6dcecd88bee67e8c9cb842fb" gracePeriod=30
Oct 31 07:46:59.163575 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.163551    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kube-rbac-proxy" containerID="cri-o://9b11252e2155b452d3b77061412e1e1f584c94cbd1a3696572ce8b1ae0be31d8" gracePeriod=30
Oct 31 07:46:59.163777 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.163525    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent" containerID="cri-o://c0fd344e525b200e55716c3e9ae3fc8ba2998c1678e601444ad70c82f2cbcdaf" gracePeriod=30
Oct 31 07:46:59.244518 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.244475    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"]
Oct 31 07:46:59.244804 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.244746    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9" podUID="c4f1cbd9-37df-4c8b-b162-e73acd77a326" containerName="kserve-container" containerID="cri-o://d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9" gracePeriod=30
Oct 31 07:46:59.244944 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.244779    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9" podUID="c4f1cbd9-37df-4c8b-b162-e73acd77a326" containerName="kube-rbac-proxy" containerID="cri-o://d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7" gracePeriod=30
Oct 31 07:46:59.498031 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.498009    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:46:59.574334 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.574309    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c4f1cbd9-37df-4c8b-b162-e73acd77a326-proxy-tls\" (UniqueName: \"kubernetes.io/secret/c4f1cbd9-37df-4c8b-b162-e73acd77a326-proxy-tls\") pod \"c4f1cbd9-37df-4c8b-b162-e73acd77a326\" (UID: \"c4f1cbd9-37df-4c8b-b162-e73acd77a326\") "
Oct 31 07:46:59.574482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.574400    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4f1cbd9-37df-4c8b-b162-e73acd77a326-kube-api-access-zpl4v\" (UniqueName: \"kubernetes.io/projected/c4f1cbd9-37df-4c8b-b162-e73acd77a326-kube-api-access-zpl4v\") pod \"c4f1cbd9-37df-4c8b-b162-e73acd77a326\" (UID: \"c4f1cbd9-37df-4c8b-b162-e73acd77a326\") "
Oct 31 07:46:59.574482 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.574449    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4f1cbd9-37df-4c8b-b162-e73acd77a326-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c4f1cbd9-37df-4c8b-b162-e73acd77a326-kube-rbac-proxy-sar-config\") pod \"c4f1cbd9-37df-4c8b-b162-e73acd77a326\" (UID: \"c4f1cbd9-37df-4c8b-b162-e73acd77a326\") "
Oct 31 07:46:59.574818 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.574794    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4f1cbd9-37df-4c8b-b162-e73acd77a326-kube-rbac-proxy-sar-config" pod "c4f1cbd9-37df-4c8b-b162-e73acd77a326" (UID: "c4f1cbd9-37df-4c8b-b162-e73acd77a326"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:46:59.576440 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.576411    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4f1cbd9-37df-4c8b-b162-e73acd77a326-proxy-tls" pod "c4f1cbd9-37df-4c8b-b162-e73acd77a326" (UID: "c4f1cbd9-37df-4c8b-b162-e73acd77a326"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:46:59.576558 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.576411    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4f1cbd9-37df-4c8b-b162-e73acd77a326-kube-api-access-zpl4v" pod "c4f1cbd9-37df-4c8b-b162-e73acd77a326" (UID: "c4f1cbd9-37df-4c8b-b162-e73acd77a326"). InnerVolumeSpecName "kube-api-access-zpl4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:46:59.651101 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.651072    4322 generic.go:356] "Generic (PLEG): container finished" podID="c4f1cbd9-37df-4c8b-b162-e73acd77a326" containerID="d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7" exitCode=0
Oct 31 07:46:59.651101 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.651094    4322 generic.go:356] "Generic (PLEG): container finished" podID="c4f1cbd9-37df-4c8b-b162-e73acd77a326" containerID="d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9" exitCode=2
Oct 31 07:46:59.651581 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.651124    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"
Oct 31 07:46:59.651581 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.651166    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9" event={"ID":"c4f1cbd9-37df-4c8b-b162-e73acd77a326","Type":"ContainerDied","Data":"d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7"}
Oct 31 07:46:59.651581 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.651245    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9" event={"ID":"c4f1cbd9-37df-4c8b-b162-e73acd77a326","Type":"ContainerDied","Data":"d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9"}
Oct 31 07:46:59.651581 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.651271    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9" event={"ID":"c4f1cbd9-37df-4c8b-b162-e73acd77a326","Type":"ContainerDied","Data":"85c9bca8321e955c93f5cd4a6cffb9632cd07cea95baafa513e0cf64f4b6d90b"}
Oct 31 07:46:59.651581 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.651299    4322 scope.go:122] "RemoveContainer" containerID="d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7"
Oct 31 07:46:59.653831 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.653809    4322 generic.go:356] "Generic (PLEG): container finished" podID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerID="9b11252e2155b452d3b77061412e1e1f584c94cbd1a3696572ce8b1ae0be31d8" exitCode=0
Oct 31 07:46:59.653916 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.653877    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" event={"ID":"a8675074-6daf-4c11-9c2b-2ed73c7900b0","Type":"ContainerDied","Data":"9b11252e2155b452d3b77061412e1e1f584c94cbd1a3696572ce8b1ae0be31d8"}
Oct 31 07:46:59.660315 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.660296    4322 scope.go:122] "RemoveContainer" containerID="d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9"
Oct 31 07:46:59.668499 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.668470    4322 scope.go:122] "RemoveContainer" containerID="d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7"
Oct 31 07:46:59.668747 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:46:59.668727    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7\": container with ID starting with d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7 not found: ID does not exist" containerID="d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7"
Oct 31 07:46:59.668821 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.668760    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7"} err="failed to get container status \"d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7\": rpc error: code = NotFound desc = could not find container \"d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7\": container with ID starting with d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7 not found: ID does not exist"
Oct 31 07:46:59.668821 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.668784    4322 scope.go:122] "RemoveContainer" containerID="d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9"
Oct 31 07:46:59.669047 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:46:59.669029    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9\": container with ID starting with d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9 not found: ID does not exist" containerID="d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9"
Oct 31 07:46:59.669085 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.669053    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9"} err="failed to get container status \"d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9\": rpc error: code = NotFound desc = could not find container \"d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9\": container with ID starting with d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9 not found: ID does not exist"
Oct 31 07:46:59.669085 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.669068    4322 scope.go:122] "RemoveContainer" containerID="d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7"
Oct 31 07:46:59.669303 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.669285    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7"} err="failed to get container status \"d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7\": rpc error: code = NotFound desc = could not find container \"d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7\": container with ID starting with d185204e7f0061ec253ac21595ba5651adc2b1675cae245c31f5c86382e7abe7 not found: ID does not exist"
Oct 31 07:46:59.669353 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.669304    4322 scope.go:122] "RemoveContainer" containerID="d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9"
Oct 31 07:46:59.669520 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.669502    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9"} err="failed to get container status \"d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9\": rpc error: code = NotFound desc = could not find container \"d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9\": container with ID starting with d7fbaa98a17a700b0b5f30bf6fafac93be1a8e1e53f5cc8a637cb9d320799cc9 not found: ID does not exist"
Oct 31 07:46:59.675007 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.674990    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c4f1cbd9-37df-4c8b-b162-e73acd77a326-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:46:59.675057 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.675011    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpl4v\" (UniqueName: \"kubernetes.io/projected/c4f1cbd9-37df-4c8b-b162-e73acd77a326-kube-api-access-zpl4v\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:46:59.675057 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.675021    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c4f1cbd9-37df-4c8b-b162-e73acd77a326-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:46:59.701007 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.700988    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"]
Oct 31 07:46:59.708416 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:46:59.708399    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7f5ae2e6e4/message-dumper-raw-6ca8d-predictor-6f4b7dcd54-xwkm9"]
Oct 31 07:47:00.870884 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:00.870848    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4f1cbd9-37df-4c8b-b162-e73acd77a326" path="/var/lib/kubelet/pods/c4f1cbd9-37df-4c8b-b162-e73acd77a326/volumes"
Oct 31 07:47:01.265487 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:01.265450    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.37:8643/healthz\": dial tcp 10.134.0.37:8643: connect: connection refused"
Oct 31 07:47:01.710451 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:47:01.710367    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:47:04.679090 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:04.679057    4322 generic.go:356] "Generic (PLEG): container finished" podID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerID="049c4736b4b85e17c2cda75d6678f7481645b11f6dcecd88bee67e8c9cb842fb" exitCode=0
Oct 31 07:47:04.679494 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:04.679125    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" event={"ID":"a8675074-6daf-4c11-9c2b-2ed73c7900b0","Type":"ContainerDied","Data":"049c4736b4b85e17c2cda75d6678f7481645b11f6dcecd88bee67e8c9cb842fb"}
Oct 31 07:47:06.264920 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:06.264882    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.37:8643/healthz\": dial tcp 10.134.0.37:8643: connect: connection refused"
Oct 31 07:47:06.270261 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:06.270212    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Oct 31 07:47:06.270654 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:06.270635    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:47:11.265021 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:11.264982    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.37:8643/healthz\": dial tcp 10.134.0.37:8643: connect: connection refused"
Oct 31 07:47:11.265426 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:11.265094    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:47:16.265369 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:16.265330    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.37:8643/healthz\": dial tcp 10.134.0.37:8643: connect: connection refused"
Oct 31 07:47:16.269638 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:16.269604    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Oct 31 07:47:16.269946 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:16.269923    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:47:21.264866 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:21.264827    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.37:8643/healthz\": dial tcp 10.134.0.37:8643: connect: connection refused"
Oct 31 07:47:26.265377 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:26.265329    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.37:8643/healthz\": dial tcp 10.134.0.37:8643: connect: connection refused"
Oct 31 07:47:26.269631 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:26.269598    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Oct 31 07:47:26.269769 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:26.269712    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:47:26.270027 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:26.270004    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:47:26.270113 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:26.270102    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:47:29.309132 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.309107    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:47:29.418386 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.418316    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8675074-6daf-4c11-9c2b-2ed73c7900b0-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a8675074-6daf-4c11-9c2b-2ed73c7900b0-cabundle-cert\") pod \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") "
Oct 31 07:47:29.418386 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.418361    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kserve-provision-location\") pod \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") "
Oct 31 07:47:29.418597 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.418388    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kube-rbac-proxy-sar-config\") pod \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") "
Oct 31 07:47:29.418597 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.418432    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8675074-6daf-4c11-9c2b-2ed73c7900b0-proxy-tls\" (UniqueName: \"kubernetes.io/secret/a8675074-6daf-4c11-9c2b-2ed73c7900b0-proxy-tls\") pod \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") "
Oct 31 07:47:29.418597 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.418472    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kube-api-access-mdzx5\" (UniqueName: \"kubernetes.io/projected/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kube-api-access-mdzx5\") pod \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\" (UID: \"a8675074-6daf-4c11-9c2b-2ed73c7900b0\") "
Oct 31 07:47:29.418758 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.418692    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kserve-provision-location" pod "a8675074-6daf-4c11-9c2b-2ed73c7900b0" (UID: "a8675074-6daf-4c11-9c2b-2ed73c7900b0"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:47:29.418758 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.418704    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8675074-6daf-4c11-9c2b-2ed73c7900b0-cabundle-cert" pod "a8675074-6daf-4c11-9c2b-2ed73c7900b0" (UID: "a8675074-6daf-4c11-9c2b-2ed73c7900b0"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:47:29.418834 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.418775    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kube-rbac-proxy-sar-config" pod "a8675074-6daf-4c11-9c2b-2ed73c7900b0" (UID: "a8675074-6daf-4c11-9c2b-2ed73c7900b0"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:47:29.420532 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.420510    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8675074-6daf-4c11-9c2b-2ed73c7900b0-proxy-tls" pod "a8675074-6daf-4c11-9c2b-2ed73c7900b0" (UID: "a8675074-6daf-4c11-9c2b-2ed73c7900b0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:47:29.420621 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.420507    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kube-api-access-mdzx5" pod "a8675074-6daf-4c11-9c2b-2ed73c7900b0" (UID: "a8675074-6daf-4c11-9c2b-2ed73c7900b0"). InnerVolumeSpecName "kube-api-access-mdzx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:47:29.519489 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.519459    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a8675074-6daf-4c11-9c2b-2ed73c7900b0-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:47:29.519489 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.519484    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdzx5\" (UniqueName: \"kubernetes.io/projected/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kube-api-access-mdzx5\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:47:29.519660 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.519498    4322 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a8675074-6daf-4c11-9c2b-2ed73c7900b0-cabundle-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:47:29.519660 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.519511    4322 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kserve-provision-location\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:47:29.519660 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.519523    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a8675074-6daf-4c11-9c2b-2ed73c7900b0-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:47:29.783795 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.783758    4322 generic.go:356] "Generic (PLEG): container finished" podID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerID="c0fd344e525b200e55716c3e9ae3fc8ba2998c1678e601444ad70c82f2cbcdaf" exitCode=0
Oct 31 07:47:29.783971 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.783828    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"
Oct 31 07:47:29.783971 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.783838    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" event={"ID":"a8675074-6daf-4c11-9c2b-2ed73c7900b0","Type":"ContainerDied","Data":"c0fd344e525b200e55716c3e9ae3fc8ba2998c1678e601444ad70c82f2cbcdaf"}
Oct 31 07:47:29.783971 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.783926    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw" event={"ID":"a8675074-6daf-4c11-9c2b-2ed73c7900b0","Type":"ContainerDied","Data":"23fdd8e92ba28a6bcae0a6093f653f29c5f58eb084b4b4c78c33f9197e273719"}
Oct 31 07:47:29.784139 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.783971    4322 scope.go:122] "RemoveContainer" containerID="c0fd344e525b200e55716c3e9ae3fc8ba2998c1678e601444ad70c82f2cbcdaf"
Oct 31 07:47:29.793201 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.793181    4322 scope.go:122] "RemoveContainer" containerID="9b11252e2155b452d3b77061412e1e1f584c94cbd1a3696572ce8b1ae0be31d8"
Oct 31 07:47:29.800303 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.800286    4322 scope.go:122] "RemoveContainer" containerID="049c4736b4b85e17c2cda75d6678f7481645b11f6dcecd88bee67e8c9cb842fb"
Oct 31 07:47:29.807592 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.807576    4322 scope.go:122] "RemoveContainer" containerID="f27765ad68c109a49468128b2de6dd8ab485040ec823a0cae2d5f697cd58b6cd"
Oct 31 07:47:29.814493 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.814479    4322 scope.go:122] "RemoveContainer" containerID="c0fd344e525b200e55716c3e9ae3fc8ba2998c1678e601444ad70c82f2cbcdaf"
Oct 31 07:47:29.814733 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:47:29.814714    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0fd344e525b200e55716c3e9ae3fc8ba2998c1678e601444ad70c82f2cbcdaf\": container with ID starting with c0fd344e525b200e55716c3e9ae3fc8ba2998c1678e601444ad70c82f2cbcdaf not found: ID does not exist" containerID="c0fd344e525b200e55716c3e9ae3fc8ba2998c1678e601444ad70c82f2cbcdaf"
Oct 31 07:47:29.814793 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.814739    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0fd344e525b200e55716c3e9ae3fc8ba2998c1678e601444ad70c82f2cbcdaf"} err="failed to get container status \"c0fd344e525b200e55716c3e9ae3fc8ba2998c1678e601444ad70c82f2cbcdaf\": rpc error: code = NotFound desc = could not find container \"c0fd344e525b200e55716c3e9ae3fc8ba2998c1678e601444ad70c82f2cbcdaf\": container with ID starting with c0fd344e525b200e55716c3e9ae3fc8ba2998c1678e601444ad70c82f2cbcdaf not found: ID does not exist"
Oct 31 07:47:29.814793 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.814753    4322 scope.go:122] "RemoveContainer" containerID="9b11252e2155b452d3b77061412e1e1f584c94cbd1a3696572ce8b1ae0be31d8"
Oct 31 07:47:29.814986 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:47:29.814971    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b11252e2155b452d3b77061412e1e1f584c94cbd1a3696572ce8b1ae0be31d8\": container with ID starting with 9b11252e2155b452d3b77061412e1e1f584c94cbd1a3696572ce8b1ae0be31d8 not found: ID does not exist" containerID="9b11252e2155b452d3b77061412e1e1f584c94cbd1a3696572ce8b1ae0be31d8"
Oct 31 07:47:29.815028 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.814999    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b11252e2155b452d3b77061412e1e1f584c94cbd1a3696572ce8b1ae0be31d8"} err="failed to get container status \"9b11252e2155b452d3b77061412e1e1f584c94cbd1a3696572ce8b1ae0be31d8\": rpc error: code = NotFound desc = could not find container \"9b11252e2155b452d3b77061412e1e1f584c94cbd1a3696572ce8b1ae0be31d8\": container with ID starting with 9b11252e2155b452d3b77061412e1e1f584c94cbd1a3696572ce8b1ae0be31d8 not found: ID does not exist"
Oct 31 07:47:29.815028 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.815012    4322 scope.go:122] "RemoveContainer" containerID="049c4736b4b85e17c2cda75d6678f7481645b11f6dcecd88bee67e8c9cb842fb"
Oct 31 07:47:29.815198 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:47:29.815183    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"049c4736b4b85e17c2cda75d6678f7481645b11f6dcecd88bee67e8c9cb842fb\": container with ID starting with 049c4736b4b85e17c2cda75d6678f7481645b11f6dcecd88bee67e8c9cb842fb not found: ID does not exist" containerID="049c4736b4b85e17c2cda75d6678f7481645b11f6dcecd88bee67e8c9cb842fb"
Oct 31 07:47:29.815250 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.815203    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049c4736b4b85e17c2cda75d6678f7481645b11f6dcecd88bee67e8c9cb842fb"} err="failed to get container status \"049c4736b4b85e17c2cda75d6678f7481645b11f6dcecd88bee67e8c9cb842fb\": rpc error: code = NotFound desc = could not find container \"049c4736b4b85e17c2cda75d6678f7481645b11f6dcecd88bee67e8c9cb842fb\": container with ID starting with 049c4736b4b85e17c2cda75d6678f7481645b11f6dcecd88bee67e8c9cb842fb not found: ID does not exist"
Oct 31 07:47:29.815250 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.815216    4322 scope.go:122] "RemoveContainer" containerID="f27765ad68c109a49468128b2de6dd8ab485040ec823a0cae2d5f697cd58b6cd"
Oct 31 07:47:29.815412 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:47:29.815396    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f27765ad68c109a49468128b2de6dd8ab485040ec823a0cae2d5f697cd58b6cd\": container with ID starting with f27765ad68c109a49468128b2de6dd8ab485040ec823a0cae2d5f697cd58b6cd not found: ID does not exist" containerID="f27765ad68c109a49468128b2de6dd8ab485040ec823a0cae2d5f697cd58b6cd"
Oct 31 07:47:29.815453 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.815416    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f27765ad68c109a49468128b2de6dd8ab485040ec823a0cae2d5f697cd58b6cd"} err="failed to get container status \"f27765ad68c109a49468128b2de6dd8ab485040ec823a0cae2d5f697cd58b6cd\": rpc error: code = NotFound desc = could not find container \"f27765ad68c109a49468128b2de6dd8ab485040ec823a0cae2d5f697cd58b6cd\": container with ID starting with f27765ad68c109a49468128b2de6dd8ab485040ec823a0cae2d5f697cd58b6cd not found: ID does not exist"
Oct 31 07:47:29.837252 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.837204    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"]
Oct 31 07:47:29.848705 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:29.848683    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-logger-raw-6ca8d-predictor-f5bd9f5f7-b4cdw"]
Oct 31 07:47:30.870610 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:30.870575    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" path="/var/lib/kubelet/pods/a8675074-6daf-4c11-9c2b-2ed73c7900b0/volumes"
Oct 31 07:47:31.213824 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.213747    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"]
Oct 31 07:47:31.214097 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214084    4322 cpu_manager.go:414] "containerMap: removing container" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="storage-initializer"
Oct 31 07:47:31.214174 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214097    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="storage-initializer"
Oct 31 07:47:31.214174 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214111    4322 cpu_manager.go:414] "containerMap: removing container" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent"
Oct 31 07:47:31.214174 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214116    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent"
Oct 31 07:47:31.214174 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214125    4322 cpu_manager.go:414] "containerMap: removing container" podUID="c4f1cbd9-37df-4c8b-b162-e73acd77a326" containerName="kserve-container"
Oct 31 07:47:31.214174 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214135    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4f1cbd9-37df-4c8b-b162-e73acd77a326" containerName="kserve-container"
Oct 31 07:47:31.214174 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214159    4322 cpu_manager.go:414] "containerMap: removing container" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kube-rbac-proxy"
Oct 31 07:47:31.214174 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214166    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kube-rbac-proxy"
Oct 31 07:47:31.214435 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214179    4322 cpu_manager.go:414] "containerMap: removing container" podUID="c4f1cbd9-37df-4c8b-b162-e73acd77a326" containerName="kube-rbac-proxy"
Oct 31 07:47:31.214435 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214186    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4f1cbd9-37df-4c8b-b162-e73acd77a326" containerName="kube-rbac-proxy"
Oct 31 07:47:31.214435 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214201    4322 cpu_manager.go:414] "containerMap: removing container" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container"
Oct 31 07:47:31.214435 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214209    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container"
Oct 31 07:47:31.214435 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214293    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="agent"
Oct 31 07:47:31.214435 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214303    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4f1cbd9-37df-4c8b-b162-e73acd77a326" containerName="kserve-container"
Oct 31 07:47:31.214435 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214310    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kube-rbac-proxy"
Oct 31 07:47:31.214435 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214317    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4f1cbd9-37df-4c8b-b162-e73acd77a326" containerName="kube-rbac-proxy"
Oct 31 07:47:31.214435 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.214323    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8675074-6daf-4c11-9c2b-2ed73c7900b0" containerName="kserve-container"
Oct 31 07:47:31.219099 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.219081    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.233757 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.233726    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"kube-root-ca.crt\""
Oct 31 07:47:31.245621 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.245595    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 07:47:31.245621 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.245595    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"openshift-service-ca.crt\""
Oct 31 07:47:31.245808 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.245631    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"isvc-sklearn-scale-raw-e9f0c-kube-rbac-proxy-sar-config\""
Oct 31 07:47:31.245923 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.245867    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"isvc-sklearn-scale-raw-e9f0c-predictor-serving-cert\""
Oct 31 07:47:31.245923 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.245892    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"seaweedfs-s3-creds\""
Oct 31 07:47:31.246124 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.246004    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7f5ae2e6e4\"/\"default-dockercfg-d6nzh\""
Oct 31 07:47:31.247406 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.247385    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"]
Oct 31 07:47:31.332940 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.332891    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.332940 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.332941    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-proxy-tls\") pod \"isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.333166 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.332997    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qnq6\" (UniqueName: \"kubernetes.io/projected/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kube-api-access-7qnq6\") pod \"isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.333166 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.333038    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.333166 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.333065    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-cabundle-cert\") pod \"isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.433977 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.433931    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.434134 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.433993    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-cabundle-cert\") pod \"isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.434134 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.434070    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.434134 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.434098    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-proxy-tls\") pod \"isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.434134 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.434121    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qnq6\" (UniqueName: \"kubernetes.io/projected/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kube-api-access-7qnq6\") pod \"isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.434491 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.434466    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.434699 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.434681    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.434772 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.434755    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-cabundle-cert\") pod \"isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.436532 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.436516    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-proxy-tls\") pod \"isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.456875 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.456850    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qnq6\" (UniqueName: \"kubernetes.io/projected/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kube-api-access-7qnq6\") pod \"isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") " pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.535578 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.535550    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:31.709449 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:47:31.709412    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:47:31.837945 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:31.837878    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"]
Oct 31 07:47:31.841003 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:47:31.840975    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56ae86a4_55a3_4884_b0e3_7b1f8d55e25e.slice/crio-09e4a7a9863f5c3d52e8efa5d8fb76fb46411127bbc2df5da054f3d2e1b2e5aa WatchSource:0}: Error finding container 09e4a7a9863f5c3d52e8efa5d8fb76fb46411127bbc2df5da054f3d2e1b2e5aa: Status 404 returned error can't find the container with id 09e4a7a9863f5c3d52e8efa5d8fb76fb46411127bbc2df5da054f3d2e1b2e5aa
Oct 31 07:47:32.800114 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:32.800078    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" event={"ID":"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e","Type":"ContainerStarted","Data":"ccf18f4d26d2b947dbbb9210b0834a24589a5b8be6578a467beaef8ef519fcc2"}
Oct 31 07:47:32.800630 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:32.800147    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" event={"ID":"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e","Type":"ContainerStarted","Data":"09e4a7a9863f5c3d52e8efa5d8fb76fb46411127bbc2df5da054f3d2e1b2e5aa"}
Oct 31 07:47:35.814843 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:35.814817    4322 generic.go:356] "Generic (PLEG): container finished" podID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerID="ccf18f4d26d2b947dbbb9210b0834a24589a5b8be6578a467beaef8ef519fcc2" exitCode=0
Oct 31 07:47:35.815140 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:35.814851    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" event={"ID":"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e","Type":"ContainerDied","Data":"ccf18f4d26d2b947dbbb9210b0834a24589a5b8be6578a467beaef8ef519fcc2"}
Oct 31 07:47:36.820967 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:36.820932    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" event={"ID":"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e","Type":"ContainerStarted","Data":"5255c5d100f2e4c575a6b554ab0085b6e67807c76f048f0fd611316bce378525"}
Oct 31 07:47:36.821489 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:36.821003    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" event={"ID":"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e","Type":"ContainerStarted","Data":"a5847db10888053ba5ea1fc2a450201c379607e8f5b1e44f291fb38ce68ba4c0"}
Oct 31 07:47:36.821489 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:36.821251    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:36.879319 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:36.879262    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podStartSLOduration=1.97883498 podStartE2EDuration="5.879245625s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.900410645s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:47:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:47:36.878594072 +0000 UTC m=+1098.142414694" watchObservedRunningTime="2026-10-31 07:47:36.879245625 +0000 UTC m=+1098.143066235"
Oct 31 07:47:37.825598 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:37.825570    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:37.826843 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:37.826815    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:47:38.829601 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:38.829556    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:47:43.833929 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:43.833895    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:47:43.834429 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:43.834401    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:47:53.835262 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:47:53.835205    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:48:01.710101 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:48:01.710017    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:48:03.834537 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:48:03.834499    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:48:13.834759 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:48:13.834721    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:48:23.835075 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:48:23.835038    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:48:31.710790 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:48:31.710752    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:48:33.834937 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:48:33.834899    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:48:43.834988 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:48:43.834947    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:48:53.834848 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:48:53.834798    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:48:57.866404 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:48:57.866363    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:49:01.709860 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:49:01.709825    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:49:07.866334 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:07.866293    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:49:15.919442 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:15.919409    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"]
Oct 31 07:49:15.923583 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:15.923564    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:15.951684 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:15.951655    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"]
Oct 31 07:49:15.993541 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:15.993510    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kserve-provision-location\") pod \"isvc-canary-create-predictor-66d784d8dd-5tjzw\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") " pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:15.993684 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:15.993555    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-66d784d8dd-5tjzw\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") " pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:15.993737 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:15.993725    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-cabundle-cert\") pod \"isvc-canary-create-predictor-66d784d8dd-5tjzw\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") " pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:15.993829 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:15.993809    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9kdc\" (UniqueName: \"kubernetes.io/projected/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kube-api-access-m9kdc\") pod \"isvc-canary-create-predictor-66d784d8dd-5tjzw\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") " pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:15.993907 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:15.993894    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-proxy-tls\") pod \"isvc-canary-create-predictor-66d784d8dd-5tjzw\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") " pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:16.018911 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.018876    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7281997251\"/\"seaweedfs-s3-creds\""
Oct 31 07:49:16.019034 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.018981    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"openshift-service-ca.crt\""
Oct 31 07:49:16.019034 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.018988    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 07:49:16.019157 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.019043    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7281997251\"/\"default-dockercfg-t4zp9\""
Oct 31 07:49:16.019157 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.019039    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"kube-root-ca.crt\""
Oct 31 07:49:16.019157 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.019054    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"isvc-canary-create-kube-rbac-proxy-sar-config\""
Oct 31 07:49:16.019157 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.019134    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7281997251\"/\"isvc-canary-create-predictor-serving-cert\""
Oct 31 07:49:16.094723 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.094698    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kserve-provision-location\") pod \"isvc-canary-create-predictor-66d784d8dd-5tjzw\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") " pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:16.094888 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.094736    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-66d784d8dd-5tjzw\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") " pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:16.094888 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.094758    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-cabundle-cert\") pod \"isvc-canary-create-predictor-66d784d8dd-5tjzw\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") " pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:16.094888 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.094785    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9kdc\" (UniqueName: \"kubernetes.io/projected/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kube-api-access-m9kdc\") pod \"isvc-canary-create-predictor-66d784d8dd-5tjzw\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") " pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:16.094888 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.094850    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-proxy-tls\") pod \"isvc-canary-create-predictor-66d784d8dd-5tjzw\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") " pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:16.095143 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.095118    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kserve-provision-location\") pod \"isvc-canary-create-predictor-66d784d8dd-5tjzw\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") " pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:16.095646 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.095621    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-cabundle-cert\") pod \"isvc-canary-create-predictor-66d784d8dd-5tjzw\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") " pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:16.095739 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.095628    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-66d784d8dd-5tjzw\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") " pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:16.097199 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.097178    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-proxy-tls\") pod \"isvc-canary-create-predictor-66d784d8dd-5tjzw\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") " pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:16.117440 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.117415    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9kdc\" (UniqueName: \"kubernetes.io/projected/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kube-api-access-m9kdc\") pod \"isvc-canary-create-predictor-66d784d8dd-5tjzw\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") " pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:16.241411 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.241332    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:16.544065 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:16.544036    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"]
Oct 31 07:49:16.547404 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:49:16.547381    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee19fed6_b00e_49d3_8ae2_d9bc1d88d432.slice/crio-e57de6ff635a434c5442afd899a38d7f4cc98103c5c34b8bbc2afebd48832eeb WatchSource:0}: Error finding container e57de6ff635a434c5442afd899a38d7f4cc98103c5c34b8bbc2afebd48832eeb: Status 404 returned error can't find the container with id e57de6ff635a434c5442afd899a38d7f4cc98103c5c34b8bbc2afebd48832eeb
Oct 31 07:49:17.237218 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:17.237186    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" event={"ID":"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432","Type":"ContainerStarted","Data":"6b42ef93087eae5ddf6e4914bc0c4732c7ea8c39586fd3653c8224bad94b0839"}
Oct 31 07:49:17.237597 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:17.237267    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" event={"ID":"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432","Type":"ContainerStarted","Data":"e57de6ff635a434c5442afd899a38d7f4cc98103c5c34b8bbc2afebd48832eeb"}
Oct 31 07:49:17.866582 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:17.866541    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:49:21.256605 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:21.256575    4322 generic.go:356] "Generic (PLEG): container finished" podID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerID="6b42ef93087eae5ddf6e4914bc0c4732c7ea8c39586fd3653c8224bad94b0839" exitCode=0
Oct 31 07:49:21.256989 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:21.256642    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" event={"ID":"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432","Type":"ContainerDied","Data":"6b42ef93087eae5ddf6e4914bc0c4732c7ea8c39586fd3653c8224bad94b0839"}
Oct 31 07:49:22.262764 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:22.262729    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" event={"ID":"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432","Type":"ContainerStarted","Data":"d55fa93d66a766bb9290d0e6796999caca8624a7d3df78c600b2b63aaf98ea7e"}
Oct 31 07:49:22.263159 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:22.262793    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" event={"ID":"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432","Type":"ContainerStarted","Data":"0077973b2f6a227d1ca5eccc5b3bc9c5771c15851bdcb8398006129b81e94ace"}
Oct 31 07:49:22.263159 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:22.262982    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:22.319327 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:22.319284    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" podStartSLOduration=3.369680902 podStartE2EDuration="7.319271096s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.949590194s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:49:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:49:22.317783668 +0000 UTC m=+1203.581604278" watchObservedRunningTime="2026-10-31 07:49:22.319271096 +0000 UTC m=+1203.583091706"
Oct 31 07:49:23.266561 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:23.266527    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:23.267906 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:23.267881    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Oct 31 07:49:24.270280 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:24.270222    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Oct 31 07:49:27.866715 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:27.866677    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:49:29.274411 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:29.274381    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:49:29.274807 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:29.274771    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Oct 31 07:49:31.710668 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:49:31.710589    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:49:37.866581 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:37.866532    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:49:39.274861 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:39.274823    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Oct 31 07:49:47.866550 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:47.866507    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:49:49.274899 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:49.274860    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Oct 31 07:49:57.866781 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:57.866741    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:49:59.275631 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:49:59.275597    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Oct 31 07:50:01.710278 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:50:01.710215    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:50:05.865936 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:05.865898    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:50:09.274867 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:09.274825    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Oct 31 07:50:15.866411 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:15.866381    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:50:19.274853 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:19.274816    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Oct 31 07:50:22.615725 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:22.615693    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"]
Oct 31 07:50:22.616324 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:22.615973    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" containerID="cri-o://a5847db10888053ba5ea1fc2a450201c379607e8f5b1e44f291fb38ce68ba4c0" gracePeriod=30
Oct 31 07:50:22.616324 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:22.615996    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kube-rbac-proxy" containerID="cri-o://5255c5d100f2e4c575a6b554ab0085b6e67807c76f048f0fd611316bce378525" gracePeriod=30
Oct 31 07:50:23.529780 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:23.529746    4322 generic.go:356] "Generic (PLEG): container finished" podID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerID="5255c5d100f2e4c575a6b554ab0085b6e67807c76f048f0fd611316bce378525" exitCode=0
Oct 31 07:50:23.529952 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:23.529820    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" event={"ID":"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e","Type":"ContainerDied","Data":"5255c5d100f2e4c575a6b554ab0085b6e67807c76f048f0fd611316bce378525"}
Oct 31 07:50:23.830678 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:23.830590    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.38:8643/healthz\": dial tcp 10.134.0.38:8643: connect: connection refused"
Oct 31 07:50:25.866287 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:25.866220    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Oct 31 07:50:28.830467 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:28.830428    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.38:8643/healthz\": dial tcp 10.134.0.38:8643: connect: connection refused"
Oct 31 07:50:29.274875 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:29.274832    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Oct 31 07:50:31.709866 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:50:31.709828    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:50:33.830498 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:33.830461    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.38:8643/healthz\": dial tcp 10.134.0.38:8643: connect: connection refused"
Oct 31 07:50:33.830896 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:33.830576    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:50:35.353848 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.353823    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:50:35.476512 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.476430    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kube-rbac-proxy-sar-config\") pod \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") "
Oct 31 07:50:35.476512 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.476471    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-cabundle-cert\") pod \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") "
Oct 31 07:50:35.476512 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.476494    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-proxy-tls\" (UniqueName: \"kubernetes.io/secret/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-proxy-tls\") pod \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") "
Oct 31 07:50:35.476736 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.476556    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kserve-provision-location\") pod \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") "
Oct 31 07:50:35.476736 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.476581    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kube-api-access-7qnq6\" (UniqueName: \"kubernetes.io/projected/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kube-api-access-7qnq6\") pod \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\" (UID: \"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e\") "
Oct 31 07:50:35.476896 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.476871    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kserve-provision-location" pod "56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" (UID: "56ae86a4-55a3-4884-b0e3-7b1f8d55e25e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:50:35.476955 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.476844    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kube-rbac-proxy-sar-config" pod "56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" (UID: "56ae86a4-55a3-4884-b0e3-7b1f8d55e25e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:50:35.476994 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.476980    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-cabundle-cert" pod "56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" (UID: "56ae86a4-55a3-4884-b0e3-7b1f8d55e25e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:50:35.478622 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.478592    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kube-api-access-7qnq6" pod "56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" (UID: "56ae86a4-55a3-4884-b0e3-7b1f8d55e25e"). InnerVolumeSpecName "kube-api-access-7qnq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:50:35.478732 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.478676    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-proxy-tls" pod "56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" (UID: "56ae86a4-55a3-4884-b0e3-7b1f8d55e25e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:50:35.577598 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.577566    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:50:35.577598 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.577593    4322 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-cabundle-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:50:35.577598 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.577604    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:50:35.577825 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.577613    4322 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kserve-provision-location\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:50:35.577825 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.577623    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qnq6\" (UniqueName: \"kubernetes.io/projected/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e-kube-api-access-7qnq6\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:50:35.585697 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.585666    4322 generic.go:356] "Generic (PLEG): container finished" podID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerID="a5847db10888053ba5ea1fc2a450201c379607e8f5b1e44f291fb38ce68ba4c0" exitCode=0
Oct 31 07:50:35.585850 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.585724    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"
Oct 31 07:50:35.585850 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.585742    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" event={"ID":"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e","Type":"ContainerDied","Data":"a5847db10888053ba5ea1fc2a450201c379607e8f5b1e44f291fb38ce68ba4c0"}
Oct 31 07:50:35.585850 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.585811    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs" event={"ID":"56ae86a4-55a3-4884-b0e3-7b1f8d55e25e","Type":"ContainerDied","Data":"09e4a7a9863f5c3d52e8efa5d8fb76fb46411127bbc2df5da054f3d2e1b2e5aa"}
Oct 31 07:50:35.585850 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.585847    4322 scope.go:122] "RemoveContainer" containerID="5255c5d100f2e4c575a6b554ab0085b6e67807c76f048f0fd611316bce378525"
Oct 31 07:50:35.594562 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.594547    4322 scope.go:122] "RemoveContainer" containerID="a5847db10888053ba5ea1fc2a450201c379607e8f5b1e44f291fb38ce68ba4c0"
Oct 31 07:50:35.601961 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.601943    4322 scope.go:122] "RemoveContainer" containerID="ccf18f4d26d2b947dbbb9210b0834a24589a5b8be6578a467beaef8ef519fcc2"
Oct 31 07:50:35.609136 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.609116    4322 scope.go:122] "RemoveContainer" containerID="5255c5d100f2e4c575a6b554ab0085b6e67807c76f048f0fd611316bce378525"
Oct 31 07:50:35.609424 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:50:35.609406    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5255c5d100f2e4c575a6b554ab0085b6e67807c76f048f0fd611316bce378525\": container with ID starting with 5255c5d100f2e4c575a6b554ab0085b6e67807c76f048f0fd611316bce378525 not found: ID does not exist" containerID="5255c5d100f2e4c575a6b554ab0085b6e67807c76f048f0fd611316bce378525"
Oct 31 07:50:35.609486 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.609432    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5255c5d100f2e4c575a6b554ab0085b6e67807c76f048f0fd611316bce378525"} err="failed to get container status \"5255c5d100f2e4c575a6b554ab0085b6e67807c76f048f0fd611316bce378525\": rpc error: code = NotFound desc = could not find container \"5255c5d100f2e4c575a6b554ab0085b6e67807c76f048f0fd611316bce378525\": container with ID starting with 5255c5d100f2e4c575a6b554ab0085b6e67807c76f048f0fd611316bce378525 not found: ID does not exist"
Oct 31 07:50:35.609486 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.609450    4322 scope.go:122] "RemoveContainer" containerID="a5847db10888053ba5ea1fc2a450201c379607e8f5b1e44f291fb38ce68ba4c0"
Oct 31 07:50:35.609680 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:50:35.609662    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5847db10888053ba5ea1fc2a450201c379607e8f5b1e44f291fb38ce68ba4c0\": container with ID starting with a5847db10888053ba5ea1fc2a450201c379607e8f5b1e44f291fb38ce68ba4c0 not found: ID does not exist" containerID="a5847db10888053ba5ea1fc2a450201c379607e8f5b1e44f291fb38ce68ba4c0"
Oct 31 07:50:35.609723 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.609686    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5847db10888053ba5ea1fc2a450201c379607e8f5b1e44f291fb38ce68ba4c0"} err="failed to get container status \"a5847db10888053ba5ea1fc2a450201c379607e8f5b1e44f291fb38ce68ba4c0\": rpc error: code = NotFound desc = could not find container \"a5847db10888053ba5ea1fc2a450201c379607e8f5b1e44f291fb38ce68ba4c0\": container with ID starting with a5847db10888053ba5ea1fc2a450201c379607e8f5b1e44f291fb38ce68ba4c0 not found: ID does not exist"
Oct 31 07:50:35.609723 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.609701    4322 scope.go:122] "RemoveContainer" containerID="ccf18f4d26d2b947dbbb9210b0834a24589a5b8be6578a467beaef8ef519fcc2"
Oct 31 07:50:35.609934 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:50:35.609919    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccf18f4d26d2b947dbbb9210b0834a24589a5b8be6578a467beaef8ef519fcc2\": container with ID starting with ccf18f4d26d2b947dbbb9210b0834a24589a5b8be6578a467beaef8ef519fcc2 not found: ID does not exist" containerID="ccf18f4d26d2b947dbbb9210b0834a24589a5b8be6578a467beaef8ef519fcc2"
Oct 31 07:50:35.609973 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.609937    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccf18f4d26d2b947dbbb9210b0834a24589a5b8be6578a467beaef8ef519fcc2"} err="failed to get container status \"ccf18f4d26d2b947dbbb9210b0834a24589a5b8be6578a467beaef8ef519fcc2\": rpc error: code = NotFound desc = could not find container \"ccf18f4d26d2b947dbbb9210b0834a24589a5b8be6578a467beaef8ef519fcc2\": container with ID starting with ccf18f4d26d2b947dbbb9210b0834a24589a5b8be6578a467beaef8ef519fcc2 not found: ID does not exist"
Oct 31 07:50:35.636221 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.636196    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"]
Oct 31 07:50:35.639177 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:35.639152    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7f5ae2e6e4/isvc-sklearn-scale-raw-e9f0c-predictor-5c6b9bcf59-bxcvs"]
Oct 31 07:50:36.871032 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:36.871000    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" path="/var/lib/kubelet/pods/56ae86a4-55a3-4884-b0e3-7b1f8d55e25e/volumes"
Oct 31 07:50:39.275437 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:39.275408    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:50:51.000438 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:51.000403    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"]
Oct 31 07:50:51.001522 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:51.000798    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kserve-container" containerID="cri-o://0077973b2f6a227d1ca5eccc5b3bc9c5771c15851bdcb8398006129b81e94ace" gracePeriod=30
Oct 31 07:50:51.001522 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:51.001343    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kube-rbac-proxy" containerID="cri-o://d55fa93d66a766bb9290d0e6796999caca8624a7d3df78c600b2b63aaf98ea7e" gracePeriod=30
Oct 31 07:50:51.658380 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:51.658345    4322 generic.go:356] "Generic (PLEG): container finished" podID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerID="d55fa93d66a766bb9290d0e6796999caca8624a7d3df78c600b2b63aaf98ea7e" exitCode=0
Oct 31 07:50:51.658552 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:51.658419    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" event={"ID":"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432","Type":"ContainerDied","Data":"d55fa93d66a766bb9290d0e6796999caca8624a7d3df78c600b2b63aaf98ea7e"}
Oct 31 07:50:54.271185 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:54.271139    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.39:8643/healthz\": dial tcp 10.134.0.39:8643: connect: connection refused"
Oct 31 07:50:55.849570 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:55.849545    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:50:55.940506 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:55.940423    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-proxy-tls\") pod \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") "
Oct 31 07:50:55.940506 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:55.940469    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-cabundle-cert\") pod \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") "
Oct 31 07:50:55.940717 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:55.940533    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kube-api-access-m9kdc\" (UniqueName: \"kubernetes.io/projected/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kube-api-access-m9kdc\") pod \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") "
Oct 31 07:50:55.940717 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:55.940556    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kserve-provision-location\") pod \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") "
Oct 31 07:50:55.940822 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:55.940734    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kube-rbac-proxy-sar-config\") pod \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\" (UID: \"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432\") "
Oct 31 07:50:55.940989 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:55.940877    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kserve-provision-location" pod "ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" (UID: "ee19fed6-b00e-49d3-8ae2-d9bc1d88d432"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:50:55.941112 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:55.940892    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-cabundle-cert" pod "ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" (UID: "ee19fed6-b00e-49d3-8ae2-d9bc1d88d432"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:50:55.941112 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:55.941010    4322 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kserve-provision-location\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:50:55.941112 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:55.941055    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kube-rbac-proxy-sar-config" pod "ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" (UID: "ee19fed6-b00e-49d3-8ae2-d9bc1d88d432"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:50:55.942637 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:55.942618    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kube-api-access-m9kdc" pod "ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" (UID: "ee19fed6-b00e-49d3-8ae2-d9bc1d88d432"). InnerVolumeSpecName "kube-api-access-m9kdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:50:55.942716 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:55.942697    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-proxy-tls" pod "ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" (UID: "ee19fed6-b00e-49d3-8ae2-d9bc1d88d432"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:50:56.042341 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.042303    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:50:56.042341 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.042335    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:50:56.042341 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.042344    4322 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-cabundle-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:50:56.042341 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.042353    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9kdc\" (UniqueName: \"kubernetes.io/projected/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432-kube-api-access-m9kdc\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:50:56.680942 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.680908    4322 generic.go:356] "Generic (PLEG): container finished" podID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerID="0077973b2f6a227d1ca5eccc5b3bc9c5771c15851bdcb8398006129b81e94ace" exitCode=0
Oct 31 07:50:56.681115 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.680968    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"
Oct 31 07:50:56.681115 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.680988    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" event={"ID":"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432","Type":"ContainerDied","Data":"0077973b2f6a227d1ca5eccc5b3bc9c5771c15851bdcb8398006129b81e94ace"}
Oct 31 07:50:56.681115 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.681045    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw" event={"ID":"ee19fed6-b00e-49d3-8ae2-d9bc1d88d432","Type":"ContainerDied","Data":"e57de6ff635a434c5442afd899a38d7f4cc98103c5c34b8bbc2afebd48832eeb"}
Oct 31 07:50:56.681115 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.681070    4322 scope.go:122] "RemoveContainer" containerID="d55fa93d66a766bb9290d0e6796999caca8624a7d3df78c600b2b63aaf98ea7e"
Oct 31 07:50:56.689700 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.689683    4322 scope.go:122] "RemoveContainer" containerID="0077973b2f6a227d1ca5eccc5b3bc9c5771c15851bdcb8398006129b81e94ace"
Oct 31 07:50:56.696955 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.696938    4322 scope.go:122] "RemoveContainer" containerID="6b42ef93087eae5ddf6e4914bc0c4732c7ea8c39586fd3653c8224bad94b0839"
Oct 31 07:50:56.704034 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.704017    4322 scope.go:122] "RemoveContainer" containerID="d55fa93d66a766bb9290d0e6796999caca8624a7d3df78c600b2b63aaf98ea7e"
Oct 31 07:50:56.704318 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:50:56.704300    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d55fa93d66a766bb9290d0e6796999caca8624a7d3df78c600b2b63aaf98ea7e\": container with ID starting with d55fa93d66a766bb9290d0e6796999caca8624a7d3df78c600b2b63aaf98ea7e not found: ID does not exist" containerID="d55fa93d66a766bb9290d0e6796999caca8624a7d3df78c600b2b63aaf98ea7e"
Oct 31 07:50:56.704376 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.704328    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d55fa93d66a766bb9290d0e6796999caca8624a7d3df78c600b2b63aaf98ea7e"} err="failed to get container status \"d55fa93d66a766bb9290d0e6796999caca8624a7d3df78c600b2b63aaf98ea7e\": rpc error: code = NotFound desc = could not find container \"d55fa93d66a766bb9290d0e6796999caca8624a7d3df78c600b2b63aaf98ea7e\": container with ID starting with d55fa93d66a766bb9290d0e6796999caca8624a7d3df78c600b2b63aaf98ea7e not found: ID does not exist"
Oct 31 07:50:56.704376 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.704345    4322 scope.go:122] "RemoveContainer" containerID="0077973b2f6a227d1ca5eccc5b3bc9c5771c15851bdcb8398006129b81e94ace"
Oct 31 07:50:56.704585 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:50:56.704567    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0077973b2f6a227d1ca5eccc5b3bc9c5771c15851bdcb8398006129b81e94ace\": container with ID starting with 0077973b2f6a227d1ca5eccc5b3bc9c5771c15851bdcb8398006129b81e94ace not found: ID does not exist" containerID="0077973b2f6a227d1ca5eccc5b3bc9c5771c15851bdcb8398006129b81e94ace"
Oct 31 07:50:56.704654 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.704596    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0077973b2f6a227d1ca5eccc5b3bc9c5771c15851bdcb8398006129b81e94ace"} err="failed to get container status \"0077973b2f6a227d1ca5eccc5b3bc9c5771c15851bdcb8398006129b81e94ace\": rpc error: code = NotFound desc = could not find container \"0077973b2f6a227d1ca5eccc5b3bc9c5771c15851bdcb8398006129b81e94ace\": container with ID starting with 0077973b2f6a227d1ca5eccc5b3bc9c5771c15851bdcb8398006129b81e94ace not found: ID does not exist"
Oct 31 07:50:56.704654 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.704620    4322 scope.go:122] "RemoveContainer" containerID="6b42ef93087eae5ddf6e4914bc0c4732c7ea8c39586fd3653c8224bad94b0839"
Oct 31 07:50:56.704855 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:50:56.704835    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b42ef93087eae5ddf6e4914bc0c4732c7ea8c39586fd3653c8224bad94b0839\": container with ID starting with 6b42ef93087eae5ddf6e4914bc0c4732c7ea8c39586fd3653c8224bad94b0839 not found: ID does not exist" containerID="6b42ef93087eae5ddf6e4914bc0c4732c7ea8c39586fd3653c8224bad94b0839"
Oct 31 07:50:56.704899 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.704862    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b42ef93087eae5ddf6e4914bc0c4732c7ea8c39586fd3653c8224bad94b0839"} err="failed to get container status \"6b42ef93087eae5ddf6e4914bc0c4732c7ea8c39586fd3653c8224bad94b0839\": rpc error: code = NotFound desc = could not find container \"6b42ef93087eae5ddf6e4914bc0c4732c7ea8c39586fd3653c8224bad94b0839\": container with ID starting with 6b42ef93087eae5ddf6e4914bc0c4732c7ea8c39586fd3653c8224bad94b0839 not found: ID does not exist"
Oct 31 07:50:56.743646 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.743623    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"]
Oct 31 07:50:56.755575 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.755553    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-create-predictor-66d784d8dd-5tjzw"]
Oct 31 07:50:56.870370 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:56.870340    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" path="/var/lib/kubelet/pods/ee19fed6-b00e-49d3-8ae2-d9bc1d88d432/volumes"
Oct 31 07:50:58.914990 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.914956    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"]
Oct 31 07:50:58.916780 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.916743    4322 cpu_manager.go:414] "containerMap: removing container" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kserve-container"
Oct 31 07:50:58.916780 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.916776    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kserve-container"
Oct 31 07:50:58.916995 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.916794    4322 cpu_manager.go:414] "containerMap: removing container" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="storage-initializer"
Oct 31 07:50:58.916995 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.916804    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="storage-initializer"
Oct 31 07:50:58.916995 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.916827    4322 cpu_manager.go:414] "containerMap: removing container" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kube-rbac-proxy"
Oct 31 07:50:58.916995 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.916835    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kube-rbac-proxy"
Oct 31 07:50:58.916995 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.916865    4322 cpu_manager.go:414] "containerMap: removing container" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container"
Oct 31 07:50:58.916995 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.916873    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container"
Oct 31 07:50:58.916995 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.916909    4322 cpu_manager.go:414] "containerMap: removing container" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kube-rbac-proxy"
Oct 31 07:50:58.916995 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.916916    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kube-rbac-proxy"
Oct 31 07:50:58.916995 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.916939    4322 cpu_manager.go:414] "containerMap: removing container" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="storage-initializer"
Oct 31 07:50:58.916995 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.916947    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="storage-initializer"
Oct 31 07:50:58.917583 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.917103    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kserve-container"
Oct 31 07:50:58.917583 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.917116    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee19fed6-b00e-49d3-8ae2-d9bc1d88d432" containerName="kube-rbac-proxy"
Oct 31 07:50:58.917583 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.917136    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kserve-container"
Oct 31 07:50:58.917583 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.917148    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="56ae86a4-55a3-4884-b0e3-7b1f8d55e25e" containerName="kube-rbac-proxy"
Oct 31 07:50:58.923294 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.923269    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:58.953448 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.953410    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"]
Oct 31 07:50:58.968331 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.968305    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3bdc5c74-708e-490b-b093-fda12a4c174e-proxy-tls\") pod \"isvc-canary-promote-predictor-574fc57cbd-p6gzz\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") " pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:58.968452 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.968341    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3bdc5c74-708e-490b-b093-fda12a4c174e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-574fc57cbd-p6gzz\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") " pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:58.968452 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.968362    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zdp4\" (UniqueName: \"kubernetes.io/projected/3bdc5c74-708e-490b-b093-fda12a4c174e-kube-api-access-2zdp4\") pod \"isvc-canary-promote-predictor-574fc57cbd-p6gzz\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") " pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:58.968529 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.968507    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3bdc5c74-708e-490b-b093-fda12a4c174e-cabundle-cert\") pod \"isvc-canary-promote-predictor-574fc57cbd-p6gzz\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") " pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:58.968650 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:58.968623    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3bdc5c74-708e-490b-b093-fda12a4c174e-kserve-provision-location\") pod \"isvc-canary-promote-predictor-574fc57cbd-p6gzz\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") " pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:59.013018 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.012994    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\""
Oct 31 07:50:59.013018 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.013013    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7281997251\"/\"default-dockercfg-t4zp9\""
Oct 31 07:50:59.013193 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.013015    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7281997251\"/\"seaweedfs-s3-creds\""
Oct 31 07:50:59.013193 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.013015    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 07:50:59.013193 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.013029    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"openshift-service-ca.crt\""
Oct 31 07:50:59.013193 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.013078    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7281997251\"/\"kube-root-ca.crt\""
Oct 31 07:50:59.013193 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.013151    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7281997251\"/\"isvc-canary-promote-predictor-serving-cert\""
Oct 31 07:50:59.069744 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.069718    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3bdc5c74-708e-490b-b093-fda12a4c174e-kserve-provision-location\") pod \"isvc-canary-promote-predictor-574fc57cbd-p6gzz\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") " pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:59.069924 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.069756    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3bdc5c74-708e-490b-b093-fda12a4c174e-proxy-tls\") pod \"isvc-canary-promote-predictor-574fc57cbd-p6gzz\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") " pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:59.069924 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.069776    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3bdc5c74-708e-490b-b093-fda12a4c174e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-574fc57cbd-p6gzz\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") " pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:59.069924 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.069801    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zdp4\" (UniqueName: \"kubernetes.io/projected/3bdc5c74-708e-490b-b093-fda12a4c174e-kube-api-access-2zdp4\") pod \"isvc-canary-promote-predictor-574fc57cbd-p6gzz\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") " pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:59.069924 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.069843    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3bdc5c74-708e-490b-b093-fda12a4c174e-cabundle-cert\") pod \"isvc-canary-promote-predictor-574fc57cbd-p6gzz\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") " pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:59.070196 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.070173    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3bdc5c74-708e-490b-b093-fda12a4c174e-kserve-provision-location\") pod \"isvc-canary-promote-predictor-574fc57cbd-p6gzz\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") " pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:59.070523 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.070495    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3bdc5c74-708e-490b-b093-fda12a4c174e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-574fc57cbd-p6gzz\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") " pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:59.070643 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.070557    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3bdc5c74-708e-490b-b093-fda12a4c174e-cabundle-cert\") pod \"isvc-canary-promote-predictor-574fc57cbd-p6gzz\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") " pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:59.072241 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.072210    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3bdc5c74-708e-490b-b093-fda12a4c174e-proxy-tls\") pod \"isvc-canary-promote-predictor-574fc57cbd-p6gzz\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") " pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:59.092287 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.092258    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zdp4\" (UniqueName: \"kubernetes.io/projected/3bdc5c74-708e-490b-b093-fda12a4c174e-kube-api-access-2zdp4\") pod \"isvc-canary-promote-predictor-574fc57cbd-p6gzz\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") " pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:59.240598 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.240513    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:50:59.546694 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.546662    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"]
Oct 31 07:50:59.550274 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:50:59.550242    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bdc5c74_708e_490b_b093_fda12a4c174e.slice/crio-c47eaa2f65b7fa1faab03f0fee505637a9295404be1a4927bddbd2374292679e WatchSource:0}: Error finding container c47eaa2f65b7fa1faab03f0fee505637a9295404be1a4927bddbd2374292679e: Status 404 returned error can't find the container with id c47eaa2f65b7fa1faab03f0fee505637a9295404be1a4927bddbd2374292679e
Oct 31 07:50:59.696948 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.696916    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" event={"ID":"3bdc5c74-708e-490b-b093-fda12a4c174e","Type":"ContainerStarted","Data":"89ee7d33da1fdb0e2e05d06dd0bbe53c8c8ef6007b2c1376918c137abf1922d2"}
Oct 31 07:50:59.697110 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:50:59.696976    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" event={"ID":"3bdc5c74-708e-490b-b093-fda12a4c174e","Type":"ContainerStarted","Data":"c47eaa2f65b7fa1faab03f0fee505637a9295404be1a4927bddbd2374292679e"}
Oct 31 07:51:01.711439 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:51:01.711357    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:51:03.716745 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:51:03.716663    4322 generic.go:356] "Generic (PLEG): container finished" podID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerID="89ee7d33da1fdb0e2e05d06dd0bbe53c8c8ef6007b2c1376918c137abf1922d2" exitCode=0
Oct 31 07:51:03.716745 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:51:03.716733    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" event={"ID":"3bdc5c74-708e-490b-b093-fda12a4c174e","Type":"ContainerDied","Data":"89ee7d33da1fdb0e2e05d06dd0bbe53c8c8ef6007b2c1376918c137abf1922d2"}
Oct 31 07:51:04.723885 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:51:04.723846    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" event={"ID":"3bdc5c74-708e-490b-b093-fda12a4c174e","Type":"ContainerStarted","Data":"7ba603a4105bda49647c3a2b21fd6ed8c5a291708143b264a07f3b106f0f19aa"}
Oct 31 07:51:04.724333 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:51:04.723930    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" event={"ID":"3bdc5c74-708e-490b-b093-fda12a4c174e","Type":"ContainerStarted","Data":"899db3d4262e88100e12d3a993c91540d74f63d881cb9ac6335f7be955578060"}
Oct 31 07:51:04.724333 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:51:04.724207    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:51:04.724468 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:51:04.724374    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:51:04.725648 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:51:04.725626    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 31 07:51:04.780018 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:51:04.779968    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" podStartSLOduration=3.059779774 podStartE2EDuration="6.779956762s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.720176988s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:50:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:51:04.778529175 +0000 UTC m=+1306.042349768" watchObservedRunningTime="2026-10-31 07:51:04.779956762 +0000 UTC m=+1306.043777373"
Oct 31 07:51:05.729119 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:51:05.729081    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 31 07:51:10.733589 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:51:10.733563    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:51:10.734025 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:51:10.734000    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 31 07:51:20.734171 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:51:20.734127    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 31 07:51:30.734002 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:51:30.733967    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 31 07:51:31.710206 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:51:31.710172    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:51:40.734513 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:51:40.734471    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 31 07:51:50.734246 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:51:50.734203    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 31 07:52:00.733975 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:00.733938    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 31 07:52:01.711116 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:52:01.711076    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:52:10.734310 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:10.734274    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 31 07:52:13.866265 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:13.866204    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 31 07:52:23.866946 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:23.866903    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:52:31.610959 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:31.610844    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"]
Oct 31 07:52:31.611506 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:31.611199    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kserve-container" containerID="cri-o://899db3d4262e88100e12d3a993c91540d74f63d881cb9ac6335f7be955578060" gracePeriod=30
Oct 31 07:52:31.611506 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:31.611254    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kube-rbac-proxy" containerID="cri-o://7ba603a4105bda49647c3a2b21fd6ed8c5a291708143b264a07f3b106f0f19aa" gracePeriod=30
Oct 31 07:52:31.710073 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:52:31.710037    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:52:32.101723 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:32.101692    4322 generic.go:356] "Generic (PLEG): container finished" podID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerID="7ba603a4105bda49647c3a2b21fd6ed8c5a291708143b264a07f3b106f0f19aa" exitCode=0
Oct 31 07:52:32.101879 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:32.101762    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" event={"ID":"3bdc5c74-708e-490b-b093-fda12a4c174e","Type":"ContainerDied","Data":"7ba603a4105bda49647c3a2b21fd6ed8c5a291708143b264a07f3b106f0f19aa"}
Oct 31 07:52:33.866189 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:33.866145    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Oct 31 07:52:34.287102 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.287068    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"]
Oct 31 07:52:34.291049 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.291027    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.305483 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.305461    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7281997251\"/\"isvc-canary-promote-v2-predictor-serving-cert\""
Oct 31 07:52:34.319404 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.319379    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"]
Oct 31 07:52:34.464680 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.464642    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3889f80f-3e7c-4295-b247-91b91870312b-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") " pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.464680 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.464686    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3889f80f-3e7c-4295-b247-91b91870312b-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") " pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.464919 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.464775    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3889f80f-3e7c-4295-b247-91b91870312b-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") " pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.464919 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.464819    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shjvc\" (UniqueName: \"kubernetes.io/projected/3889f80f-3e7c-4295-b247-91b91870312b-kube-api-access-shjvc\") pod \"isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") " pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.464919 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.464877    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3889f80f-3e7c-4295-b247-91b91870312b-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") " pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.566089 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.565994    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3889f80f-3e7c-4295-b247-91b91870312b-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") " pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.566089 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.566047    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3889f80f-3e7c-4295-b247-91b91870312b-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") " pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.566089 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.566073    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shjvc\" (UniqueName: \"kubernetes.io/projected/3889f80f-3e7c-4295-b247-91b91870312b-kube-api-access-shjvc\") pod \"isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") " pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.566421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.566111    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3889f80f-3e7c-4295-b247-91b91870312b-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") " pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.566421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.566195    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3889f80f-3e7c-4295-b247-91b91870312b-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") " pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.566542 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.566497    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3889f80f-3e7c-4295-b247-91b91870312b-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") " pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.566802 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.566781    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3889f80f-3e7c-4295-b247-91b91870312b-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") " pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.566846 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.566805    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3889f80f-3e7c-4295-b247-91b91870312b-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") " pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.568649 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.568620    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3889f80f-3e7c-4295-b247-91b91870312b-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") " pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.589103 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.589075    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shjvc\" (UniqueName: \"kubernetes.io/projected/3889f80f-3e7c-4295-b247-91b91870312b-kube-api-access-shjvc\") pod \"isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") " pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.607818 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.607789    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:34.911708 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:34.911620    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"]
Oct 31 07:52:34.914914 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:52:34.914887    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3889f80f_3e7c_4295_b247_91b91870312b.slice/crio-49e092513c3ca3edd4c380b7b83f7c9c423d36d915330be51c98da993edd7273 WatchSource:0}: Error finding container 49e092513c3ca3edd4c380b7b83f7c9c423d36d915330be51c98da993edd7273: Status 404 returned error can't find the container with id 49e092513c3ca3edd4c380b7b83f7c9c423d36d915330be51c98da993edd7273
Oct 31 07:52:35.116198 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:35.116167    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" event={"ID":"3889f80f-3e7c-4295-b247-91b91870312b","Type":"ContainerStarted","Data":"4d20caad3ef2837572fb58b704d28148c4e4e153a5496293011338bdf0d5b8f2"}
Oct 31 07:52:35.116373 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:35.116254    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" event={"ID":"3889f80f-3e7c-4295-b247-91b91870312b","Type":"ContainerStarted","Data":"49e092513c3ca3edd4c380b7b83f7c9c423d36d915330be51c98da993edd7273"}
Oct 31 07:52:35.729705 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:35.729665    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.40:8643/healthz\": dial tcp 10.134.0.40:8643: connect: connection refused"
Oct 31 07:52:36.454668 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.454642    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:52:36.583042 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.583014    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3bdc5c74-708e-490b-b093-fda12a4c174e-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3bdc5c74-708e-490b-b093-fda12a4c174e-cabundle-cert\") pod \"3bdc5c74-708e-490b-b093-fda12a4c174e\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") "
Oct 31 07:52:36.583212 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.583101    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3bdc5c74-708e-490b-b093-fda12a4c174e-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3bdc5c74-708e-490b-b093-fda12a4c174e-kube-rbac-proxy-sar-config\") pod \"3bdc5c74-708e-490b-b093-fda12a4c174e\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") "
Oct 31 07:52:36.583212 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.583123    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3bdc5c74-708e-490b-b093-fda12a4c174e-proxy-tls\" (UniqueName: \"kubernetes.io/secret/3bdc5c74-708e-490b-b093-fda12a4c174e-proxy-tls\") pod \"3bdc5c74-708e-490b-b093-fda12a4c174e\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") "
Oct 31 07:52:36.583212 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.583144    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3bdc5c74-708e-490b-b093-fda12a4c174e-kube-api-access-2zdp4\" (UniqueName: \"kubernetes.io/projected/3bdc5c74-708e-490b-b093-fda12a4c174e-kube-api-access-2zdp4\") pod \"3bdc5c74-708e-490b-b093-fda12a4c174e\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") "
Oct 31 07:52:36.583212 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.583183    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bdc5c74-708e-490b-b093-fda12a4c174e-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3bdc5c74-708e-490b-b093-fda12a4c174e-kserve-provision-location\") pod \"3bdc5c74-708e-490b-b093-fda12a4c174e\" (UID: \"3bdc5c74-708e-490b-b093-fda12a4c174e\") "
Oct 31 07:52:36.583466 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.583427    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bdc5c74-708e-490b-b093-fda12a4c174e-cabundle-cert" pod "3bdc5c74-708e-490b-b093-fda12a4c174e" (UID: "3bdc5c74-708e-490b-b093-fda12a4c174e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:52:36.583609 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.583588    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bdc5c74-708e-490b-b093-fda12a4c174e-kserve-provision-location" pod "3bdc5c74-708e-490b-b093-fda12a4c174e" (UID: "3bdc5c74-708e-490b-b093-fda12a4c174e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:52:36.583668 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.583593    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bdc5c74-708e-490b-b093-fda12a4c174e-kube-rbac-proxy-sar-config" pod "3bdc5c74-708e-490b-b093-fda12a4c174e" (UID: "3bdc5c74-708e-490b-b093-fda12a4c174e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:52:36.585203 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.585182    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bdc5c74-708e-490b-b093-fda12a4c174e-proxy-tls" pod "3bdc5c74-708e-490b-b093-fda12a4c174e" (UID: "3bdc5c74-708e-490b-b093-fda12a4c174e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:52:36.585353 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.585255    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bdc5c74-708e-490b-b093-fda12a4c174e-kube-api-access-2zdp4" pod "3bdc5c74-708e-490b-b093-fda12a4c174e" (UID: "3bdc5c74-708e-490b-b093-fda12a4c174e"). InnerVolumeSpecName "kube-api-access-2zdp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:52:36.684162 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.684130    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3bdc5c74-708e-490b-b093-fda12a4c174e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:52:36.684162 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.684157    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3bdc5c74-708e-490b-b093-fda12a4c174e-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:52:36.684362 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.684171    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zdp4\" (UniqueName: \"kubernetes.io/projected/3bdc5c74-708e-490b-b093-fda12a4c174e-kube-api-access-2zdp4\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:52:36.684362 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.684183    4322 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3bdc5c74-708e-490b-b093-fda12a4c174e-kserve-provision-location\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:52:36.684362 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:36.684195    4322 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3bdc5c74-708e-490b-b093-fda12a4c174e-cabundle-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:52:37.126848 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:37.126813    4322 generic.go:356] "Generic (PLEG): container finished" podID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerID="899db3d4262e88100e12d3a993c91540d74f63d881cb9ac6335f7be955578060" exitCode=0
Oct 31 07:52:37.127019 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:37.126870    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"
Oct 31 07:52:37.127019 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:37.126900    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" event={"ID":"3bdc5c74-708e-490b-b093-fda12a4c174e","Type":"ContainerDied","Data":"899db3d4262e88100e12d3a993c91540d74f63d881cb9ac6335f7be955578060"}
Oct 31 07:52:37.127019 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:37.126956    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz" event={"ID":"3bdc5c74-708e-490b-b093-fda12a4c174e","Type":"ContainerDied","Data":"c47eaa2f65b7fa1faab03f0fee505637a9295404be1a4927bddbd2374292679e"}
Oct 31 07:52:37.127019 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:37.126981    4322 scope.go:122] "RemoveContainer" containerID="7ba603a4105bda49647c3a2b21fd6ed8c5a291708143b264a07f3b106f0f19aa"
Oct 31 07:52:37.135120 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:37.135099    4322 scope.go:122] "RemoveContainer" containerID="899db3d4262e88100e12d3a993c91540d74f63d881cb9ac6335f7be955578060"
Oct 31 07:52:37.142605 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:37.142588    4322 scope.go:122] "RemoveContainer" containerID="89ee7d33da1fdb0e2e05d06dd0bbe53c8c8ef6007b2c1376918c137abf1922d2"
Oct 31 07:52:37.149655 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:37.149636    4322 scope.go:122] "RemoveContainer" containerID="7ba603a4105bda49647c3a2b21fd6ed8c5a291708143b264a07f3b106f0f19aa"
Oct 31 07:52:37.149908 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:52:37.149888    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ba603a4105bda49647c3a2b21fd6ed8c5a291708143b264a07f3b106f0f19aa\": container with ID starting with 7ba603a4105bda49647c3a2b21fd6ed8c5a291708143b264a07f3b106f0f19aa not found: ID does not exist" containerID="7ba603a4105bda49647c3a2b21fd6ed8c5a291708143b264a07f3b106f0f19aa"
Oct 31 07:52:37.149961 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:37.149922    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ba603a4105bda49647c3a2b21fd6ed8c5a291708143b264a07f3b106f0f19aa"} err="failed to get container status \"7ba603a4105bda49647c3a2b21fd6ed8c5a291708143b264a07f3b106f0f19aa\": rpc error: code = NotFound desc = could not find container \"7ba603a4105bda49647c3a2b21fd6ed8c5a291708143b264a07f3b106f0f19aa\": container with ID starting with 7ba603a4105bda49647c3a2b21fd6ed8c5a291708143b264a07f3b106f0f19aa not found: ID does not exist"
Oct 31 07:52:37.149961 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:37.149943    4322 scope.go:122] "RemoveContainer" containerID="899db3d4262e88100e12d3a993c91540d74f63d881cb9ac6335f7be955578060"
Oct 31 07:52:37.150162 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:52:37.150147    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"899db3d4262e88100e12d3a993c91540d74f63d881cb9ac6335f7be955578060\": container with ID starting with 899db3d4262e88100e12d3a993c91540d74f63d881cb9ac6335f7be955578060 not found: ID does not exist" containerID="899db3d4262e88100e12d3a993c91540d74f63d881cb9ac6335f7be955578060"
Oct 31 07:52:37.150199 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:37.150166    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"899db3d4262e88100e12d3a993c91540d74f63d881cb9ac6335f7be955578060"} err="failed to get container status \"899db3d4262e88100e12d3a993c91540d74f63d881cb9ac6335f7be955578060\": rpc error: code = NotFound desc = could not find container \"899db3d4262e88100e12d3a993c91540d74f63d881cb9ac6335f7be955578060\": container with ID starting with 899db3d4262e88100e12d3a993c91540d74f63d881cb9ac6335f7be955578060 not found: ID does not exist"
Oct 31 07:52:37.150199 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:37.150179    4322 scope.go:122] "RemoveContainer" containerID="89ee7d33da1fdb0e2e05d06dd0bbe53c8c8ef6007b2c1376918c137abf1922d2"
Oct 31 07:52:37.150391 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:52:37.150375    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89ee7d33da1fdb0e2e05d06dd0bbe53c8c8ef6007b2c1376918c137abf1922d2\": container with ID starting with 89ee7d33da1fdb0e2e05d06dd0bbe53c8c8ef6007b2c1376918c137abf1922d2 not found: ID does not exist" containerID="89ee7d33da1fdb0e2e05d06dd0bbe53c8c8ef6007b2c1376918c137abf1922d2"
Oct 31 07:52:37.150427 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:37.150395    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89ee7d33da1fdb0e2e05d06dd0bbe53c8c8ef6007b2c1376918c137abf1922d2"} err="failed to get container status \"89ee7d33da1fdb0e2e05d06dd0bbe53c8c8ef6007b2c1376918c137abf1922d2\": rpc error: code = NotFound desc = could not find container \"89ee7d33da1fdb0e2e05d06dd0bbe53c8c8ef6007b2c1376918c137abf1922d2\": container with ID starting with 89ee7d33da1fdb0e2e05d06dd0bbe53c8c8ef6007b2c1376918c137abf1922d2 not found: ID does not exist"
Oct 31 07:52:37.180036 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:37.180009    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"]
Oct 31 07:52:37.192004 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:37.191980    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-promote-predictor-574fc57cbd-p6gzz"]
Oct 31 07:52:38.871075 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:38.871048    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" path="/var/lib/kubelet/pods/3bdc5c74-708e-490b-b093-fda12a4c174e/volumes"
Oct 31 07:52:39.138371 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:39.138280    4322 generic.go:356] "Generic (PLEG): container finished" podID="3889f80f-3e7c-4295-b247-91b91870312b" containerID="4d20caad3ef2837572fb58b704d28148c4e4e153a5496293011338bdf0d5b8f2" exitCode=0
Oct 31 07:52:39.138542 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:39.138356    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" event={"ID":"3889f80f-3e7c-4295-b247-91b91870312b","Type":"ContainerDied","Data":"4d20caad3ef2837572fb58b704d28148c4e4e153a5496293011338bdf0d5b8f2"}
Oct 31 07:52:40.145742 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:40.145710    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" event={"ID":"3889f80f-3e7c-4295-b247-91b91870312b","Type":"ContainerStarted","Data":"7e5f1dc0fcbb9b3cd6778b778fa98a12e298884a037da5d67464d50cb44bf916"}
Oct 31 07:52:40.146160 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:40.145774    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" event={"ID":"3889f80f-3e7c-4295-b247-91b91870312b","Type":"ContainerStarted","Data":"53a5a10f176301a9fc8efe49db0a9a95bbc3684a920086fc7706f8eacc3a6e2d"}
Oct 31 07:52:40.146160 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:40.146060    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:40.146286 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:40.146207    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:40.147484 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:40.147456    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Oct 31 07:52:40.202686 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:40.202646    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" podStartSLOduration=2.345453901 podStartE2EDuration="6.202634407s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.857180506s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:52:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:52:40.201434038 +0000 UTC m=+1401.465254649" watchObservedRunningTime="2026-10-31 07:52:40.202634407 +0000 UTC m=+1401.466455018"
Oct 31 07:52:41.149773 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:41.149735    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Oct 31 07:52:46.154073 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:46.154045    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:52:46.154553 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:46.154528    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Oct 31 07:52:56.154877 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:52:56.154840    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Oct 31 07:53:01.710012 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:53:01.709971    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:53:06.154820 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:53:06.154780    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Oct 31 07:53:16.154763 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:53:16.154723    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Oct 31 07:53:26.154639 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:53:26.154596    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Oct 31 07:53:31.710405 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:53:31.710363    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:53:36.155159 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:53:36.155121    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Oct 31 07:53:46.155123 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:53:46.155086    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Oct 31 07:53:56.155866 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:53:56.155834    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:54:01.710320 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:54:01.710242    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:54:04.717297 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:04.717263    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"]
Oct 31 07:54:04.717761 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:04.717728    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kube-rbac-proxy" containerID="cri-o://7e5f1dc0fcbb9b3cd6778b778fa98a12e298884a037da5d67464d50cb44bf916" gracePeriod=30
Oct 31 07:54:04.718674 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:04.718320    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kserve-container" containerID="cri-o://53a5a10f176301a9fc8efe49db0a9a95bbc3684a920086fc7706f8eacc3a6e2d" gracePeriod=30
Oct 31 07:54:05.509729 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:05.509692    4322 generic.go:356] "Generic (PLEG): container finished" podID="3889f80f-3e7c-4295-b247-91b91870312b" containerID="7e5f1dc0fcbb9b3cd6778b778fa98a12e298884a037da5d67464d50cb44bf916" exitCode=0
Oct 31 07:54:05.509909 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:05.509763    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" event={"ID":"3889f80f-3e7c-4295-b247-91b91870312b","Type":"ContainerDied","Data":"7e5f1dc0fcbb9b3cd6778b778fa98a12e298884a037da5d67464d50cb44bf916"}
Oct 31 07:54:06.150662 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:06.150621    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.41:8643/healthz\": dial tcp 10.134.0.41:8643: connect: connection refused"
Oct 31 07:54:06.154924 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:06.154897    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Oct 31 07:54:09.463537 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.463515    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:54:09.526867 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.526834    4322 generic.go:356] "Generic (PLEG): container finished" podID="3889f80f-3e7c-4295-b247-91b91870312b" containerID="53a5a10f176301a9fc8efe49db0a9a95bbc3684a920086fc7706f8eacc3a6e2d" exitCode=0
Oct 31 07:54:09.527029 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.526892    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"
Oct 31 07:54:09.527029 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.526913    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" event={"ID":"3889f80f-3e7c-4295-b247-91b91870312b","Type":"ContainerDied","Data":"53a5a10f176301a9fc8efe49db0a9a95bbc3684a920086fc7706f8eacc3a6e2d"}
Oct 31 07:54:09.527029 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.526973    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t" event={"ID":"3889f80f-3e7c-4295-b247-91b91870312b","Type":"ContainerDied","Data":"49e092513c3ca3edd4c380b7b83f7c9c423d36d915330be51c98da993edd7273"}
Oct 31 07:54:09.527029 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.527002    4322 scope.go:122] "RemoveContainer" containerID="7e5f1dc0fcbb9b3cd6778b778fa98a12e298884a037da5d67464d50cb44bf916"
Oct 31 07:54:09.536844 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.536819    4322 scope.go:122] "RemoveContainer" containerID="53a5a10f176301a9fc8efe49db0a9a95bbc3684a920086fc7706f8eacc3a6e2d"
Oct 31 07:54:09.544616 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.544599    4322 scope.go:122] "RemoveContainer" containerID="4d20caad3ef2837572fb58b704d28148c4e4e153a5496293011338bdf0d5b8f2"
Oct 31 07:54:09.547765 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.547748    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3889f80f-3e7c-4295-b247-91b91870312b-proxy-tls\" (UniqueName: \"kubernetes.io/secret/3889f80f-3e7c-4295-b247-91b91870312b-proxy-tls\") pod \"3889f80f-3e7c-4295-b247-91b91870312b\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") "
Oct 31 07:54:09.547842 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.547779    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3889f80f-3e7c-4295-b247-91b91870312b-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3889f80f-3e7c-4295-b247-91b91870312b-kube-rbac-proxy-sar-config\") pod \"3889f80f-3e7c-4295-b247-91b91870312b\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") "
Oct 31 07:54:09.547842 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.547802    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3889f80f-3e7c-4295-b247-91b91870312b-kube-api-access-shjvc\" (UniqueName: \"kubernetes.io/projected/3889f80f-3e7c-4295-b247-91b91870312b-kube-api-access-shjvc\") pod \"3889f80f-3e7c-4295-b247-91b91870312b\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") "
Oct 31 07:54:09.547915 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.547897    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3889f80f-3e7c-4295-b247-91b91870312b-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3889f80f-3e7c-4295-b247-91b91870312b-kserve-provision-location\") pod \"3889f80f-3e7c-4295-b247-91b91870312b\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") "
Oct 31 07:54:09.548047 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.548021    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3889f80f-3e7c-4295-b247-91b91870312b-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3889f80f-3e7c-4295-b247-91b91870312b-cabundle-cert\") pod \"3889f80f-3e7c-4295-b247-91b91870312b\" (UID: \"3889f80f-3e7c-4295-b247-91b91870312b\") "
Oct 31 07:54:09.548163 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.548123    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3889f80f-3e7c-4295-b247-91b91870312b-kube-rbac-proxy-sar-config" pod "3889f80f-3e7c-4295-b247-91b91870312b" (UID: "3889f80f-3e7c-4295-b247-91b91870312b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:54:09.548163 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.548154    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3889f80f-3e7c-4295-b247-91b91870312b-kserve-provision-location" pod "3889f80f-3e7c-4295-b247-91b91870312b" (UID: "3889f80f-3e7c-4295-b247-91b91870312b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:54:09.548420 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.548334    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3889f80f-3e7c-4295-b247-91b91870312b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:54:09.548420 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.548357    4322 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3889f80f-3e7c-4295-b247-91b91870312b-kserve-provision-location\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:54:09.548420 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.548374    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3889f80f-3e7c-4295-b247-91b91870312b-cabundle-cert" pod "3889f80f-3e7c-4295-b247-91b91870312b" (UID: "3889f80f-3e7c-4295-b247-91b91870312b"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:54:09.550268 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.550219    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3889f80f-3e7c-4295-b247-91b91870312b-proxy-tls" pod "3889f80f-3e7c-4295-b247-91b91870312b" (UID: "3889f80f-3e7c-4295-b247-91b91870312b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:54:09.550268 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.550261    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3889f80f-3e7c-4295-b247-91b91870312b-kube-api-access-shjvc" pod "3889f80f-3e7c-4295-b247-91b91870312b" (UID: "3889f80f-3e7c-4295-b247-91b91870312b"). InnerVolumeSpecName "kube-api-access-shjvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:54:09.552629 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.552608    4322 scope.go:122] "RemoveContainer" containerID="7e5f1dc0fcbb9b3cd6778b778fa98a12e298884a037da5d67464d50cb44bf916"
Oct 31 07:54:09.552903 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:54:09.552880    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e5f1dc0fcbb9b3cd6778b778fa98a12e298884a037da5d67464d50cb44bf916\": container with ID starting with 7e5f1dc0fcbb9b3cd6778b778fa98a12e298884a037da5d67464d50cb44bf916 not found: ID does not exist" containerID="7e5f1dc0fcbb9b3cd6778b778fa98a12e298884a037da5d67464d50cb44bf916"
Oct 31 07:54:09.552962 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.552911    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e5f1dc0fcbb9b3cd6778b778fa98a12e298884a037da5d67464d50cb44bf916"} err="failed to get container status \"7e5f1dc0fcbb9b3cd6778b778fa98a12e298884a037da5d67464d50cb44bf916\": rpc error: code = NotFound desc = could not find container \"7e5f1dc0fcbb9b3cd6778b778fa98a12e298884a037da5d67464d50cb44bf916\": container with ID starting with 7e5f1dc0fcbb9b3cd6778b778fa98a12e298884a037da5d67464d50cb44bf916 not found: ID does not exist"
Oct 31 07:54:09.552962 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.552933    4322 scope.go:122] "RemoveContainer" containerID="53a5a10f176301a9fc8efe49db0a9a95bbc3684a920086fc7706f8eacc3a6e2d"
Oct 31 07:54:09.553199 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:54:09.553182    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53a5a10f176301a9fc8efe49db0a9a95bbc3684a920086fc7706f8eacc3a6e2d\": container with ID starting with 53a5a10f176301a9fc8efe49db0a9a95bbc3684a920086fc7706f8eacc3a6e2d not found: ID does not exist" containerID="53a5a10f176301a9fc8efe49db0a9a95bbc3684a920086fc7706f8eacc3a6e2d"
Oct 31 07:54:09.553269 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.553205    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53a5a10f176301a9fc8efe49db0a9a95bbc3684a920086fc7706f8eacc3a6e2d"} err="failed to get container status \"53a5a10f176301a9fc8efe49db0a9a95bbc3684a920086fc7706f8eacc3a6e2d\": rpc error: code = NotFound desc = could not find container \"53a5a10f176301a9fc8efe49db0a9a95bbc3684a920086fc7706f8eacc3a6e2d\": container with ID starting with 53a5a10f176301a9fc8efe49db0a9a95bbc3684a920086fc7706f8eacc3a6e2d not found: ID does not exist"
Oct 31 07:54:09.553269 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.553221    4322 scope.go:122] "RemoveContainer" containerID="4d20caad3ef2837572fb58b704d28148c4e4e153a5496293011338bdf0d5b8f2"
Oct 31 07:54:09.553468 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:54:09.553449    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d20caad3ef2837572fb58b704d28148c4e4e153a5496293011338bdf0d5b8f2\": container with ID starting with 4d20caad3ef2837572fb58b704d28148c4e4e153a5496293011338bdf0d5b8f2 not found: ID does not exist" containerID="4d20caad3ef2837572fb58b704d28148c4e4e153a5496293011338bdf0d5b8f2"
Oct 31 07:54:09.553510 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.553471    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d20caad3ef2837572fb58b704d28148c4e4e153a5496293011338bdf0d5b8f2"} err="failed to get container status \"4d20caad3ef2837572fb58b704d28148c4e4e153a5496293011338bdf0d5b8f2\": rpc error: code = NotFound desc = could not find container \"4d20caad3ef2837572fb58b704d28148c4e4e153a5496293011338bdf0d5b8f2\": container with ID starting with 4d20caad3ef2837572fb58b704d28148c4e4e153a5496293011338bdf0d5b8f2 not found: ID does not exist"
Oct 31 07:54:09.649164 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.649137    4322 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3889f80f-3e7c-4295-b247-91b91870312b-cabundle-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:54:09.649164 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.649158    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3889f80f-3e7c-4295-b247-91b91870312b-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:54:09.649164 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.649169    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shjvc\" (UniqueName: \"kubernetes.io/projected/3889f80f-3e7c-4295-b247-91b91870312b-kube-api-access-shjvc\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:54:09.871151 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.871116    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"]
Oct 31 07:54:09.882992 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:09.882963    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7281997251/isvc-canary-promote-v2-predictor-6bc9f6df57-5vt7t"]
Oct 31 07:54:10.870938 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:10.870904    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3889f80f-3e7c-4295-b247-91b91870312b" path="/var/lib/kubelet/pods/3889f80f-3e7c-4295-b247-91b91870312b/volumes"
Oct 31 07:54:31.710005 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:54:31.709970    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:54:42.937673 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.937643    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"]
Oct 31 07:54:42.938121 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938009    4322 cpu_manager.go:414] "containerMap: removing container" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kserve-container"
Oct 31 07:54:42.938121 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938021    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kserve-container"
Oct 31 07:54:42.938121 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938033    4322 cpu_manager.go:414] "containerMap: removing container" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kube-rbac-proxy"
Oct 31 07:54:42.938121 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938039    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kube-rbac-proxy"
Oct 31 07:54:42.938121 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938047    4322 cpu_manager.go:414] "containerMap: removing container" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kube-rbac-proxy"
Oct 31 07:54:42.938121 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938053    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kube-rbac-proxy"
Oct 31 07:54:42.938121 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938059    4322 cpu_manager.go:414] "containerMap: removing container" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="storage-initializer"
Oct 31 07:54:42.938121 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938064    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="storage-initializer"
Oct 31 07:54:42.938121 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938084    4322 cpu_manager.go:414] "containerMap: removing container" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kserve-container"
Oct 31 07:54:42.938121 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938089    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kserve-container"
Oct 31 07:54:42.938121 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938103    4322 cpu_manager.go:414] "containerMap: removing container" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="storage-initializer"
Oct 31 07:54:42.938121 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938108    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="storage-initializer"
Oct 31 07:54:42.938587 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938168    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kserve-container"
Oct 31 07:54:42.938587 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938177    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kube-rbac-proxy"
Oct 31 07:54:42.938587 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938182    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="3bdc5c74-708e-490b-b093-fda12a4c174e" containerName="kube-rbac-proxy"
Oct 31 07:54:42.938587 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.938195    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="3889f80f-3e7c-4295-b247-91b91870312b" containerName="kserve-container"
Oct 31 07:54:42.941736 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.941718    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:42.956505 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.956481    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-166168988c\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 07:54:42.956505 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.956485    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-166168988c\"/\"openshift-service-ca.crt\""
Oct 31 07:54:42.956661 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.956507    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-166168988c\"/\"seaweedfs-s3-creds\""
Oct 31 07:54:42.956661 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.956486    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-166168988c\"/\"kube-root-ca.crt\""
Oct 31 07:54:42.968467 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.968416    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-166168988c\"/\"isvc-logger-raw-cipn-kube-rbac-proxy-sar-config\""
Oct 31 07:54:42.968467 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.968434    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-166168988c\"/\"isvc-logger-raw-cipn-predictor-serving-cert\""
Oct 31 07:54:42.968610 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.968585    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-166168988c\"/\"default-dockercfg-s48kc\""
Oct 31 07:54:42.970521 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:42.970499    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"]
Oct 31 07:54:43.033889 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.033855    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-cabundle-cert\") pod \"isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") " pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.034024 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.033921    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-proxy-tls\") pod \"isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") " pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.034074 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.034024    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") " pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.034109 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.034085    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kserve-provision-location\") pod \"isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") " pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.034144 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.034110    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkgcr\" (UniqueName: \"kubernetes.io/projected/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kube-api-access-tkgcr\") pod \"isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") " pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.134928 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.134898    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") " pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.135062 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.134941    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kserve-provision-location\") pod \"isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") " pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.135062 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.134960    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkgcr\" (UniqueName: \"kubernetes.io/projected/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kube-api-access-tkgcr\") pod \"isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") " pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.135062 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.135006    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-cabundle-cert\") pod \"isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") " pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.135062 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.135046    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-proxy-tls\") pod \"isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") " pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.135404 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.135378    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kserve-provision-location\") pod \"isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") " pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.135689 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.135668    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") " pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.135784 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.135671    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-cabundle-cert\") pod \"isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") " pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.137439 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.137415    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-proxy-tls\") pod \"isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") " pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.157789 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.157769    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkgcr\" (UniqueName: \"kubernetes.io/projected/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kube-api-access-tkgcr\") pod \"isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") " pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.260841 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.260818    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:43.576890 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.575387    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"]
Oct 31 07:54:43.578704 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:54:43.578676    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c0d8f9a_4cb1_485c_97b4_e2d7361d1ef5.slice/crio-a67ef73df6670368476116f7f683e84b11fc1c6712d3bbab0898057f84c2428b WatchSource:0}: Error finding container a67ef73df6670368476116f7f683e84b11fc1c6712d3bbab0898057f84c2428b: Status 404 returned error can't find the container with id a67ef73df6670368476116f7f683e84b11fc1c6712d3bbab0898057f84c2428b
Oct 31 07:54:43.679421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.679389    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" event={"ID":"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5","Type":"ContainerStarted","Data":"4d81744ef58164efe543bf71d46efa425bf23bb89c4fc538140648688997652d"}
Oct 31 07:54:43.679587 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:43.679456    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" event={"ID":"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5","Type":"ContainerStarted","Data":"a67ef73df6670368476116f7f683e84b11fc1c6712d3bbab0898057f84c2428b"}
Oct 31 07:54:48.711532 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:48.711500    4322 generic.go:356] "Generic (PLEG): container finished" podID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerID="4d81744ef58164efe543bf71d46efa425bf23bb89c4fc538140648688997652d" exitCode=0
Oct 31 07:54:48.712020 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:48.711579    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" event={"ID":"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5","Type":"ContainerDied","Data":"4d81744ef58164efe543bf71d46efa425bf23bb89c4fc538140648688997652d"}
Oct 31 07:54:49.717739 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:49.717704    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" event={"ID":"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5","Type":"ContainerStarted","Data":"6a895c8569897ca5a12feb759ce3b73d4356769b3d333ce3c8cedafe9405745f"}
Oct 31 07:54:49.718222 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:49.717786    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" event={"ID":"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5","Type":"ContainerStarted","Data":"00f118cd3e4fb6929406cddc457716a3d395dbea2052e9c2b60a34c84e401f33"}
Oct 31 07:54:49.718222 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:49.717824    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" event={"ID":"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5","Type":"ContainerStarted","Data":"9c8721f02aae88d77e17e9746a7e17bdb4bbffabcdc1c4f31cf0bc3fabf70936"}
Oct 31 07:54:49.718222 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:49.718078    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:49.787169 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:49.787116    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podStartSLOduration=3.622952699 podStartE2EDuration="7.787100712s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.164148013s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:54:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:54:49.774364596 +0000 UTC m=+1531.038185204" watchObservedRunningTime="2026-10-31 07:54:49.787100712 +0000 UTC m=+1531.050921322"
Oct 31 07:54:50.722688 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:50.722654    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:50.723068 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:50.722721    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:50.724220 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:50.724186    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 31 07:54:50.724900 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:50.724878    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:54:51.726805 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:51.726767    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 31 07:54:51.727260 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:51.727134    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:54:56.730995 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:56.730965    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:54:56.731572 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:56.731548    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 31 07:54:56.731889 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:54:56.731863    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:55:01.711497 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:55:01.711456    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:55:06.731884 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:55:06.731836    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 31 07:55:06.732434 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:55:06.732298    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:55:16.732192 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:55:16.732148    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 31 07:55:16.732745 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:55:16.732719    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:55:26.731448 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:55:26.731406    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 31 07:55:26.731922 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:55:26.731893    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:55:31.710277 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:55:31.710183    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:55:36.732058 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:55:36.732010    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 31 07:55:36.732526 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:55:36.732502    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:55:46.731611 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:55:46.731574    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 31 07:55:46.732146 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:55:46.732083    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:55:56.731710 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:55:56.731665    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 31 07:55:56.732244 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:55:56.732204    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:56:00.865785 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:00.865736    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 31 07:56:00.866219 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:00.866057    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:56:01.712894 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:56:01.712852    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:56:10.870601 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:10.870574    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:56:10.871006 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:10.870653    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:56:18.754889 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:18.754856    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"]
Oct 31 07:56:18.755449 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:18.755203    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container" containerID="cri-o://9c8721f02aae88d77e17e9746a7e17bdb4bbffabcdc1c4f31cf0bc3fabf70936" gracePeriod=30
Oct 31 07:56:18.755449 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:18.755207    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kube-rbac-proxy" containerID="cri-o://00f118cd3e4fb6929406cddc457716a3d395dbea2052e9c2b60a34c84e401f33" gracePeriod=30
Oct 31 07:56:18.755449 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:18.755211    4322 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent" containerID="cri-o://6a895c8569897ca5a12feb759ce3b73d4356769b3d333ce3c8cedafe9405745f" gracePeriod=30
Oct 31 07:56:19.092645 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:19.092611    4322 generic.go:356] "Generic (PLEG): container finished" podID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerID="00f118cd3e4fb6929406cddc457716a3d395dbea2052e9c2b60a34c84e401f33" exitCode=0
Oct 31 07:56:19.092792 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:19.092666    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" event={"ID":"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5","Type":"ContainerDied","Data":"00f118cd3e4fb6929406cddc457716a3d395dbea2052e9c2b60a34c84e401f33"}
Oct 31 07:56:20.866596 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:20.866553    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 31 07:56:20.867001 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:20.866868    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:56:21.727736 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:21.727685    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.42:8643/healthz\": dial tcp 10.134.0.42:8643: connect: connection refused"
Oct 31 07:56:24.116456 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:24.116416    4322 generic.go:356] "Generic (PLEG): container finished" podID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerID="9c8721f02aae88d77e17e9746a7e17bdb4bbffabcdc1c4f31cf0bc3fabf70936" exitCode=0
Oct 31 07:56:24.116835 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:24.116495    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" event={"ID":"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5","Type":"ContainerDied","Data":"9c8721f02aae88d77e17e9746a7e17bdb4bbffabcdc1c4f31cf0bc3fabf70936"}
Oct 31 07:56:26.727119 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:26.727073    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.42:8643/healthz\": dial tcp 10.134.0.42:8643: connect: connection refused"
Oct 31 07:56:30.865695 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:30.865655    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 31 07:56:30.866115 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:30.865962    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:56:31.710083 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:56:31.710039    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:56:31.727098 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:31.727069    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.42:8643/healthz\": dial tcp 10.134.0.42:8643: connect: connection refused"
Oct 31 07:56:31.727238 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:31.727216    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:56:36.727744 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:36.727705    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.42:8643/healthz\": dial tcp 10.134.0.42:8643: connect: connection refused"
Oct 31 07:56:40.866261 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:40.866194    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused"
Oct 31 07:56:40.866775 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:40.866633    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 07:56:40.871531 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:40.871511    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:56:40.871681 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:40.871585    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:56:41.727664 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:41.727622    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.42:8643/healthz\": dial tcp 10.134.0.42:8643: connect: connection refused"
Oct 31 07:56:46.727034 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:46.726998    4322 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.42:8643/healthz\": dial tcp 10.134.0.42:8643: connect: connection refused"
Oct 31 07:56:48.936535 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:48.936511    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:56:49.094440 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.094370    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kube-rbac-proxy-sar-config\") pod \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") "
Oct 31 07:56:49.094440 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.094420    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-cabundle-cert\") pod \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") "
Oct 31 07:56:49.094655 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.094469    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kube-api-access-tkgcr\" (UniqueName: \"kubernetes.io/projected/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kube-api-access-tkgcr\") pod \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") "
Oct 31 07:56:49.094655 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.094493    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kserve-provision-location\") pod \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") "
Oct 31 07:56:49.094655 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.094520    4322 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-proxy-tls\") pod \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\" (UID: \"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5\") "
Oct 31 07:56:49.094822 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.094785    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kube-rbac-proxy-sar-config" pod "9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" (UID: "9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:56:49.094882 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.094837    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kserve-provision-location" pod "9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" (UID: "9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:56:49.094882 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.094843    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-cabundle-cert" pod "9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" (UID: "9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:56:49.096662 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.096636    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kube-api-access-tkgcr" pod "9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" (UID: "9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5"). InnerVolumeSpecName "kube-api-access-tkgcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:56:49.096782 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.096663    4322 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-proxy-tls" pod "9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" (UID: "9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:56:49.195297 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.195273    4322 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:56:49.195297 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.195296    4322 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-cabundle-cert\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:56:49.195462 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.195308    4322 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkgcr\" (UniqueName: \"kubernetes.io/projected/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kube-api-access-tkgcr\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:56:49.195462 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.195318    4322 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-kserve-provision-location\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:56:49.195462 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.195328    4322 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5-proxy-tls\") on node \"ip-10-0-138-234.ec2.internal\" DevicePath \"\""
Oct 31 07:56:49.223502 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.223476    4322 generic.go:356] "Generic (PLEG): container finished" podID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerID="6a895c8569897ca5a12feb759ce3b73d4356769b3d333ce3c8cedafe9405745f" exitCode=137
Oct 31 07:56:49.223619 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.223524    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" event={"ID":"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5","Type":"ContainerDied","Data":"6a895c8569897ca5a12feb759ce3b73d4356769b3d333ce3c8cedafe9405745f"}
Oct 31 07:56:49.223619 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.223540    4322 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"
Oct 31 07:56:49.223619 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.223575    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb" event={"ID":"9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5","Type":"ContainerDied","Data":"a67ef73df6670368476116f7f683e84b11fc1c6712d3bbab0898057f84c2428b"}
Oct 31 07:56:49.223619 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.223605    4322 scope.go:122] "RemoveContainer" containerID="6a895c8569897ca5a12feb759ce3b73d4356769b3d333ce3c8cedafe9405745f"
Oct 31 07:56:49.232536 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.232521    4322 scope.go:122] "RemoveContainer" containerID="00f118cd3e4fb6929406cddc457716a3d395dbea2052e9c2b60a34c84e401f33"
Oct 31 07:56:49.239814 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.239800    4322 scope.go:122] "RemoveContainer" containerID="9c8721f02aae88d77e17e9746a7e17bdb4bbffabcdc1c4f31cf0bc3fabf70936"
Oct 31 07:56:49.246684 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.246667    4322 scope.go:122] "RemoveContainer" containerID="4d81744ef58164efe543bf71d46efa425bf23bb89c4fc538140648688997652d"
Oct 31 07:56:49.253795 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.253774    4322 scope.go:122] "RemoveContainer" containerID="6a895c8569897ca5a12feb759ce3b73d4356769b3d333ce3c8cedafe9405745f"
Oct 31 07:56:49.254074 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:56:49.254052    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a895c8569897ca5a12feb759ce3b73d4356769b3d333ce3c8cedafe9405745f\": container with ID starting with 6a895c8569897ca5a12feb759ce3b73d4356769b3d333ce3c8cedafe9405745f not found: ID does not exist" containerID="6a895c8569897ca5a12feb759ce3b73d4356769b3d333ce3c8cedafe9405745f"
Oct 31 07:56:49.254186 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.254081    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a895c8569897ca5a12feb759ce3b73d4356769b3d333ce3c8cedafe9405745f"} err="failed to get container status \"6a895c8569897ca5a12feb759ce3b73d4356769b3d333ce3c8cedafe9405745f\": rpc error: code = NotFound desc = could not find container \"6a895c8569897ca5a12feb759ce3b73d4356769b3d333ce3c8cedafe9405745f\": container with ID starting with 6a895c8569897ca5a12feb759ce3b73d4356769b3d333ce3c8cedafe9405745f not found: ID does not exist"
Oct 31 07:56:49.254186 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.254098    4322 scope.go:122] "RemoveContainer" containerID="00f118cd3e4fb6929406cddc457716a3d395dbea2052e9c2b60a34c84e401f33"
Oct 31 07:56:49.254399 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:56:49.254376    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00f118cd3e4fb6929406cddc457716a3d395dbea2052e9c2b60a34c84e401f33\": container with ID starting with 00f118cd3e4fb6929406cddc457716a3d395dbea2052e9c2b60a34c84e401f33 not found: ID does not exist" containerID="00f118cd3e4fb6929406cddc457716a3d395dbea2052e9c2b60a34c84e401f33"
Oct 31 07:56:49.254481 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.254407    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00f118cd3e4fb6929406cddc457716a3d395dbea2052e9c2b60a34c84e401f33"} err="failed to get container status \"00f118cd3e4fb6929406cddc457716a3d395dbea2052e9c2b60a34c84e401f33\": rpc error: code = NotFound desc = could not find container \"00f118cd3e4fb6929406cddc457716a3d395dbea2052e9c2b60a34c84e401f33\": container with ID starting with 00f118cd3e4fb6929406cddc457716a3d395dbea2052e9c2b60a34c84e401f33 not found: ID does not exist"
Oct 31 07:56:49.254481 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.254429    4322 scope.go:122] "RemoveContainer" containerID="9c8721f02aae88d77e17e9746a7e17bdb4bbffabcdc1c4f31cf0bc3fabf70936"
Oct 31 07:56:49.254711 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:56:49.254689    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c8721f02aae88d77e17e9746a7e17bdb4bbffabcdc1c4f31cf0bc3fabf70936\": container with ID starting with 9c8721f02aae88d77e17e9746a7e17bdb4bbffabcdc1c4f31cf0bc3fabf70936 not found: ID does not exist" containerID="9c8721f02aae88d77e17e9746a7e17bdb4bbffabcdc1c4f31cf0bc3fabf70936"
Oct 31 07:56:49.254792 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.254720    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c8721f02aae88d77e17e9746a7e17bdb4bbffabcdc1c4f31cf0bc3fabf70936"} err="failed to get container status \"9c8721f02aae88d77e17e9746a7e17bdb4bbffabcdc1c4f31cf0bc3fabf70936\": rpc error: code = NotFound desc = could not find container \"9c8721f02aae88d77e17e9746a7e17bdb4bbffabcdc1c4f31cf0bc3fabf70936\": container with ID starting with 9c8721f02aae88d77e17e9746a7e17bdb4bbffabcdc1c4f31cf0bc3fabf70936 not found: ID does not exist"
Oct 31 07:56:49.254792 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.254742    4322 scope.go:122] "RemoveContainer" containerID="4d81744ef58164efe543bf71d46efa425bf23bb89c4fc538140648688997652d"
Oct 31 07:56:49.254962 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:56:49.254943    4322 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d81744ef58164efe543bf71d46efa425bf23bb89c4fc538140648688997652d\": container with ID starting with 4d81744ef58164efe543bf71d46efa425bf23bb89c4fc538140648688997652d not found: ID does not exist" containerID="4d81744ef58164efe543bf71d46efa425bf23bb89c4fc538140648688997652d"
Oct 31 07:56:49.255015 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.254969    4322 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d81744ef58164efe543bf71d46efa425bf23bb89c4fc538140648688997652d"} err="failed to get container status \"4d81744ef58164efe543bf71d46efa425bf23bb89c4fc538140648688997652d\": rpc error: code = NotFound desc = could not find container \"4d81744ef58164efe543bf71d46efa425bf23bb89c4fc538140648688997652d\": container with ID starting with 4d81744ef58164efe543bf71d46efa425bf23bb89c4fc538140648688997652d not found: ID does not exist"
Oct 31 07:56:49.274807 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.274785    4322 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"]
Oct 31 07:56:49.278623 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:49.278604    4322 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-166168988c/isvc-logger-raw-cipn-predictor-569b8c8d75-q9mjb"]
Oct 31 07:56:50.876116 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:56:50.876073    4322 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" path="/var/lib/kubelet/pods/9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5/volumes"
Oct 31 07:57:01.710020 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:57:01.709934    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:57:28.939175 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.939135    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kj9n5/must-gather-mkgqt"]
Oct 31 07:57:28.939729 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.939537    4322 cpu_manager.go:414] "containerMap: removing container" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container"
Oct 31 07:57:28.939729 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.939552    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container"
Oct 31 07:57:28.939729 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.939561    4322 cpu_manager.go:414] "containerMap: removing container" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kube-rbac-proxy"
Oct 31 07:57:28.939729 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.939566    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kube-rbac-proxy"
Oct 31 07:57:28.939729 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.939579    4322 cpu_manager.go:414] "containerMap: removing container" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="storage-initializer"
Oct 31 07:57:28.939729 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.939585    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="storage-initializer"
Oct 31 07:57:28.939729 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.939599    4322 cpu_manager.go:414] "containerMap: removing container" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent"
Oct 31 07:57:28.939729 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.939606    4322 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent"
Oct 31 07:57:28.939729 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.939659    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kserve-container"
Oct 31 07:57:28.939729 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.939668    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="kube-rbac-proxy"
Oct 31 07:57:28.939729 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.939678    4322 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c0d8f9a-4cb1-485c-97b4-e2d7361d1ef5" containerName="agent"
Oct 31 07:57:28.944358 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.944338    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kj9n5/must-gather-mkgqt"
Oct 31 07:57:28.961719 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.961694    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kj9n5/must-gather-mkgqt"]
Oct 31 07:57:28.972668 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.972633    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kj9n5\"/\"kube-root-ca.crt\""
Oct 31 07:57:28.972668 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.972660    4322 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kj9n5\"/\"default-dockercfg-pcw5m\""
Oct 31 07:57:28.972811 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:28.972634    4322 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kj9n5\"/\"openshift-service-ca.crt\""
Oct 31 07:57:29.034932 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:29.034899    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/46c40eff-ceeb-4d29-9789-f9f23410e5e9-must-gather-output\") pod \"must-gather-mkgqt\" (UID: \"46c40eff-ceeb-4d29-9789-f9f23410e5e9\") " pod="openshift-must-gather-kj9n5/must-gather-mkgqt"
Oct 31 07:57:29.035062 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:29.035008    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7z49\" (UniqueName: \"kubernetes.io/projected/46c40eff-ceeb-4d29-9789-f9f23410e5e9-kube-api-access-s7z49\") pod \"must-gather-mkgqt\" (UID: \"46c40eff-ceeb-4d29-9789-f9f23410e5e9\") " pod="openshift-must-gather-kj9n5/must-gather-mkgqt"
Oct 31 07:57:29.136505 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:29.136470    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7z49\" (UniqueName: \"kubernetes.io/projected/46c40eff-ceeb-4d29-9789-f9f23410e5e9-kube-api-access-s7z49\") pod \"must-gather-mkgqt\" (UID: \"46c40eff-ceeb-4d29-9789-f9f23410e5e9\") " pod="openshift-must-gather-kj9n5/must-gather-mkgqt"
Oct 31 07:57:29.136667 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:29.136564    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/46c40eff-ceeb-4d29-9789-f9f23410e5e9-must-gather-output\") pod \"must-gather-mkgqt\" (UID: \"46c40eff-ceeb-4d29-9789-f9f23410e5e9\") " pod="openshift-must-gather-kj9n5/must-gather-mkgqt"
Oct 31 07:57:29.136910 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:29.136893    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/46c40eff-ceeb-4d29-9789-f9f23410e5e9-must-gather-output\") pod \"must-gather-mkgqt\" (UID: \"46c40eff-ceeb-4d29-9789-f9f23410e5e9\") " pod="openshift-must-gather-kj9n5/must-gather-mkgqt"
Oct 31 07:57:29.159483 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:29.159460    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7z49\" (UniqueName: \"kubernetes.io/projected/46c40eff-ceeb-4d29-9789-f9f23410e5e9-kube-api-access-s7z49\") pod \"must-gather-mkgqt\" (UID: \"46c40eff-ceeb-4d29-9789-f9f23410e5e9\") " pod="openshift-must-gather-kj9n5/must-gather-mkgqt"
Oct 31 07:57:29.270421 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:29.270392    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kj9n5/must-gather-mkgqt"
Oct 31 07:57:29.562978 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:29.562705    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kj9n5/must-gather-mkgqt"]
Oct 31 07:57:29.565713 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:57:29.565683    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46c40eff_ceeb_4d29_9789_f9f23410e5e9.slice/crio-193f94e33f34b3c0b598c88e54c1d1652c2a1f85af00522b2295f0a653b1bd63 WatchSource:0}: Error finding container 193f94e33f34b3c0b598c88e54c1d1652c2a1f85af00522b2295f0a653b1bd63: Status 404 returned error can't find the container with id 193f94e33f34b3c0b598c88e54c1d1652c2a1f85af00522b2295f0a653b1bd63
Oct 31 07:57:29.567769 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:29.567748    4322 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 07:57:30.394763 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:30.394740    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kj9n5/must-gather-mkgqt" event={"ID":"46c40eff-ceeb-4d29-9789-f9f23410e5e9","Type":"ContainerStarted","Data":"2bd102850b359a2ee94b1546174e73da0af9614fec467b8c60d73a68d49a8aea"}
Oct 31 07:57:30.395054 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:30.394778    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kj9n5/must-gather-mkgqt" event={"ID":"46c40eff-ceeb-4d29-9789-f9f23410e5e9","Type":"ContainerStarted","Data":"193f94e33f34b3c0b598c88e54c1d1652c2a1f85af00522b2295f0a653b1bd63"}
Oct 31 07:57:31.404149 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:31.404115    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kj9n5/must-gather-mkgqt" event={"ID":"46c40eff-ceeb-4d29-9789-f9f23410e5e9","Type":"ContainerStarted","Data":"45139d060f90396e17a720c781af7d937f170fe8866d066b1ac503534ebc5291"}
Oct 31 07:57:31.445113 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:31.444802    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-kj9n5/must-gather-mkgqt" podStartSLOduration=2.721570168 podStartE2EDuration="3.444783236s" totalImagesPullingTime="723.213068ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:57:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:57:31.443328605 +0000 UTC m=+1692.707149216" watchObservedRunningTime="2026-10-31 07:57:31.444783236 +0000 UTC m=+1692.708603851"
Oct 31 07:57:31.711904 ip-10-0-138-234 kubenswrapper[4322]: E1031 07:57:31.711816    4322 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-wqkwf" containerName="extractor"
Oct 31 07:57:33.530349 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:33.530320    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z9h8d_11dc8b0e-286d-42e1-bfe0-d882e7dfe073/global-pull-secret-syncer/0.log"
Oct 31 07:57:34.215002 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:34.214972    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6ll6x_e07914e4-a5e8-42b8-889a-5fab59642856/konnectivity-agent/0.log"
Oct 31 07:57:34.446791 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:34.446766    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-234.ec2.internal_700ea6542926bcf6606c986e3906f152/haproxy/0.log"
Oct 31 07:57:39.761112 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:39.761076    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cfdccab0-b4dc-474b-8300-56269ad1bf4e/alertmanager/0.log"
Oct 31 07:57:39.877868 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:39.877837    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cfdccab0-b4dc-474b-8300-56269ad1bf4e/config-reloader/0.log"
Oct 31 07:57:40.003031 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:40.003004    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cfdccab0-b4dc-474b-8300-56269ad1bf4e/kube-rbac-proxy-web/0.log"
Oct 31 07:57:40.125138 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:40.125067    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cfdccab0-b4dc-474b-8300-56269ad1bf4e/kube-rbac-proxy/0.log"
Oct 31 07:57:40.243578 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:40.243546    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cfdccab0-b4dc-474b-8300-56269ad1bf4e/kube-rbac-proxy-metric/0.log"
Oct 31 07:57:40.362142 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:40.362113    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cfdccab0-b4dc-474b-8300-56269ad1bf4e/prom-label-proxy/0.log"
Oct 31 07:57:40.480076 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:40.479987    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cfdccab0-b4dc-474b-8300-56269ad1bf4e/init-config-reloader/0.log"
Oct 31 07:57:40.746322 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:40.746217    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-8k49p_ef721f60-2404-4822-b906-44abc26d2b4f/kube-state-metrics/0.log"
Oct 31 07:57:40.866954 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:40.866925    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-8k49p_ef721f60-2404-4822-b906-44abc26d2b4f/kube-rbac-proxy-main/0.log"
Oct 31 07:57:40.986592 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:40.986567    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-8k49p_ef721f60-2404-4822-b906-44abc26d2b4f/kube-rbac-proxy-self/0.log"
Oct 31 07:57:41.109085 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:41.109056    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5b647558c9-hg4hd_64524a76-3cb4-469e-a086-e7c6fad149b1/metrics-server/0.log"
Oct 31 07:57:41.710608 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:41.710576    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-swsxq_0b806130-90d2-4c64-8c8b-f81a64560850/node-exporter/0.log"
Oct 31 07:57:41.837100 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:41.837071    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-swsxq_0b806130-90d2-4c64-8c8b-f81a64560850/kube-rbac-proxy/0.log"
Oct 31 07:57:41.955319 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:41.955296    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-swsxq_0b806130-90d2-4c64-8c8b-f81a64560850/init-textfile/0.log"
Oct 31 07:57:42.452991 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.452964    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-2vtpg_ab956a0c-6d92-4fb9-9b24-98afb9ecf44e/kube-rbac-proxy-main/0.log"
Oct 31 07:57:42.508936 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.508407    4322 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"]
Oct 31 07:57:42.515453 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.515431    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.531636 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.531611    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"]
Oct 31 07:57:42.542278 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.542246    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7c7c457-468c-4fc5-83f1-c6121205f99d-lib-modules\") pod \"perf-node-gather-daemonset-tc97z\" (UID: \"a7c7c457-468c-4fc5-83f1-c6121205f99d\") " pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.542420 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.542301    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a7c7c457-468c-4fc5-83f1-c6121205f99d-podres\") pod \"perf-node-gather-daemonset-tc97z\" (UID: \"a7c7c457-468c-4fc5-83f1-c6121205f99d\") " pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.542420 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.542329    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7c7c457-468c-4fc5-83f1-c6121205f99d-sys\") pod \"perf-node-gather-daemonset-tc97z\" (UID: \"a7c7c457-468c-4fc5-83f1-c6121205f99d\") " pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.542538 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.542448    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b69fw\" (UniqueName: \"kubernetes.io/projected/a7c7c457-468c-4fc5-83f1-c6121205f99d-kube-api-access-b69fw\") pod \"perf-node-gather-daemonset-tc97z\" (UID: \"a7c7c457-468c-4fc5-83f1-c6121205f99d\") " pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.542538 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.542484    4322 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a7c7c457-468c-4fc5-83f1-c6121205f99d-proc\") pod \"perf-node-gather-daemonset-tc97z\" (UID: \"a7c7c457-468c-4fc5-83f1-c6121205f99d\") " pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.570909 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.570878    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-2vtpg_ab956a0c-6d92-4fb9-9b24-98afb9ecf44e/kube-rbac-proxy-self/0.log"
Oct 31 07:57:42.643784 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.643751    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7c7c457-468c-4fc5-83f1-c6121205f99d-sys\") pod \"perf-node-gather-daemonset-tc97z\" (UID: \"a7c7c457-468c-4fc5-83f1-c6121205f99d\") " pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.643961 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.643810    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b69fw\" (UniqueName: \"kubernetes.io/projected/a7c7c457-468c-4fc5-83f1-c6121205f99d-kube-api-access-b69fw\") pod \"perf-node-gather-daemonset-tc97z\" (UID: \"a7c7c457-468c-4fc5-83f1-c6121205f99d\") " pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.643961 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.643841    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a7c7c457-468c-4fc5-83f1-c6121205f99d-proc\") pod \"perf-node-gather-daemonset-tc97z\" (UID: \"a7c7c457-468c-4fc5-83f1-c6121205f99d\") " pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.643961 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.643875    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7c7c457-468c-4fc5-83f1-c6121205f99d-sys\") pod \"perf-node-gather-daemonset-tc97z\" (UID: \"a7c7c457-468c-4fc5-83f1-c6121205f99d\") " pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.643961 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.643929    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a7c7c457-468c-4fc5-83f1-c6121205f99d-proc\") pod \"perf-node-gather-daemonset-tc97z\" (UID: \"a7c7c457-468c-4fc5-83f1-c6121205f99d\") " pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.643961 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.643941    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7c7c457-468c-4fc5-83f1-c6121205f99d-lib-modules\") pod \"perf-node-gather-daemonset-tc97z\" (UID: \"a7c7c457-468c-4fc5-83f1-c6121205f99d\") " pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.644146 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.643966    4322 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a7c7c457-468c-4fc5-83f1-c6121205f99d-podres\") pod \"perf-node-gather-daemonset-tc97z\" (UID: \"a7c7c457-468c-4fc5-83f1-c6121205f99d\") " pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.644146 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.644086    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7c7c457-468c-4fc5-83f1-c6121205f99d-lib-modules\") pod \"perf-node-gather-daemonset-tc97z\" (UID: \"a7c7c457-468c-4fc5-83f1-c6121205f99d\") " pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.644146 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.644090    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a7c7c457-468c-4fc5-83f1-c6121205f99d-podres\") pod \"perf-node-gather-daemonset-tc97z\" (UID: \"a7c7c457-468c-4fc5-83f1-c6121205f99d\") " pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.666794 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.666767    4322 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b69fw\" (UniqueName: \"kubernetes.io/projected/a7c7c457-468c-4fc5-83f1-c6121205f99d-kube-api-access-b69fw\") pod \"perf-node-gather-daemonset-tc97z\" (UID: \"a7c7c457-468c-4fc5-83f1-c6121205f99d\") " pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:42.690782 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.690759    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-2vtpg_ab956a0c-6d92-4fb9-9b24-98afb9ecf44e/openshift-state-metrics/0.log"
Oct 31 07:57:42.829964 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:42.829933    4322 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:43.127735 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:43.127665    4322 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"]
Oct 31 07:57:43.132105 ip-10-0-138-234 kubenswrapper[4322]: W1031 07:57:43.132079    4322 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda7c7c457_468c_4fc5_83f1_c6121205f99d.slice/crio-7fb2086436dc3ee7292b4acad726515e11d82de066278bbe8e0421d495065d26 WatchSource:0}: Error finding container 7fb2086436dc3ee7292b4acad726515e11d82de066278bbe8e0421d495065d26: Status 404 returned error can't find the container with id 7fb2086436dc3ee7292b4acad726515e11d82de066278bbe8e0421d495065d26
Oct 31 07:57:43.505892 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:43.505806    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z" event={"ID":"a7c7c457-468c-4fc5-83f1-c6121205f99d","Type":"ContainerStarted","Data":"84c2e13097f850b15ec361394f085e83735417c2312f5001eec82438c3cb46a7"}
Oct 31 07:57:43.505892 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:43.505855    4322 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z" event={"ID":"a7c7c457-468c-4fc5-83f1-c6121205f99d","Type":"ContainerStarted","Data":"7fb2086436dc3ee7292b4acad726515e11d82de066278bbe8e0421d495065d26"}
Oct 31 07:57:43.505892 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:43.505888    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:43.546400 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:43.546359    4322 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z" podStartSLOduration=1.546347216 podStartE2EDuration="1.546347216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:57:43.545389657 +0000 UTC m=+1704.809210269" watchObservedRunningTime="2026-10-31 07:57:43.546347216 +0000 UTC m=+1704.810167826"
Oct 31 07:57:43.614600 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:43.614575    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-2665c_77485de8-4d57-478d-9544-2263f942e8a6/prometheus-operator/0.log"
Oct 31 07:57:43.729676 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:43.729640    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-2665c_77485de8-4d57-478d-9544-2263f942e8a6/kube-rbac-proxy/0.log"
Oct 31 07:57:43.980176 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:43.980148    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77dbb7489d-x64wh_e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5/telemeter-client/0.log"
Oct 31 07:57:44.099961 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:44.099924    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77dbb7489d-x64wh_e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5/reload/0.log"
Oct 31 07:57:44.214011 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:44.213988    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77dbb7489d-x64wh_e642f0a5-36c7-4d08-8f2e-ba00e9dc1cb5/kube-rbac-proxy/0.log"
Oct 31 07:57:44.335146 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:44.335119    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564c757d94-dl45p_36f67793-b79a-4fa6-bb3f-676e694f263e/thanos-query/0.log"
Oct 31 07:57:44.450465 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:44.450440    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564c757d94-dl45p_36f67793-b79a-4fa6-bb3f-676e694f263e/kube-rbac-proxy-web/0.log"
Oct 31 07:57:44.565975 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:44.565951    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564c757d94-dl45p_36f67793-b79a-4fa6-bb3f-676e694f263e/kube-rbac-proxy/0.log"
Oct 31 07:57:44.681925 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:44.681856    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564c757d94-dl45p_36f67793-b79a-4fa6-bb3f-676e694f263e/prom-label-proxy/0.log"
Oct 31 07:57:44.796550 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:44.796520    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564c757d94-dl45p_36f67793-b79a-4fa6-bb3f-676e694f263e/kube-rbac-proxy-rules/0.log"
Oct 31 07:57:44.911922 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:44.911895    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-564c757d94-dl45p_36f67793-b79a-4fa6-bb3f-676e694f263e/kube-rbac-proxy-metrics/0.log"
Oct 31 07:57:48.207615 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:48.207593    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7dfcbd977f-lr5p9_8156d449-5d27-40c8-b65c-58394da0fad3/console/0.log"
Oct 31 07:57:49.526026 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:49.525996    4322 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kj9n5/perf-node-gather-daemonset-tc97z"
Oct 31 07:57:51.931049 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:51.931023    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w78n9_b638fc1f-3579-467c-ac54-e4a4bc891f9e/dns/0.log"
Oct 31 07:57:52.049095 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:52.049071    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w78n9_b638fc1f-3579-467c-ac54-e4a4bc891f9e/kube-rbac-proxy/0.log"
Oct 31 07:57:52.284621 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:52.284586    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gkl8q_b1ca5663-7cde-4b55-b971-e2afaf82fb5f/dns-node-resolver/0.log"
Oct 31 07:57:53.780070 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:53.780039    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fl4w7_62191e70-8059-4950-b636-b6c58d33b128/node-ca/0.log"
Oct 31 07:57:56.883966 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:56.883935    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vktw2_05163761-47bd-4b1b-93fc-1ef20691a84f/serve-healthcheck-canary/0.log"
Oct 31 07:57:57.902765 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:57.902737    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wqkwf_8f393f8a-63f4-41d7-aedd-d3df202eff35/kube-rbac-proxy/0.log"
Oct 31 07:57:58.021115 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:58.021084    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wqkwf_8f393f8a-63f4-41d7-aedd-d3df202eff35/exporter/0.log"
Oct 31 07:57:58.136209 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:58.136180    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wqkwf_8f393f8a-63f4-41d7-aedd-d3df202eff35/extractor/0.log"
Oct 31 07:57:58.974161 ip-10-0-138-234 kubenswrapper[4322]: I1031 07:57:58.974135    4322 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_periodic-gathering-k76m7-4srkl_deac7042-7310-4a66-98b3-88fbebb4f2f9/insights-gathering/0.log"